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FOREWORD 


The  papers  contained  herein  were  presented  at  the  Optimum  System  Synthesis 
Conference,  Wright-Patterson  AFB,  Ohio,  11  -  13  September  1962,  sponsored 
by  the  Aeronautical  Systems  Division.  The  program  chairman  was  Mr.  L.  Schwartz 
of  the  System  Optimization  Section,  Aerospace  Mechanics  Branch,  Flight  Control 
Laboratory.  Mr.  Schwartz  was  assisted  by  Lt.  R.  O.  Sickeler,  of  the  same 
section. 

The  selection  of  the  papers  and  speakers  was  the  responsibility  of  the 
individual  session  chairmen.  Prof.  C.  T.  Leondes  of  UCLA,  Prof,  D.  Graham 
of  Princeton  University,  Dr.  J.  P,  LaSalle  of  RIAS,  and  Mr.  E.  L.  Peterson  of 
the  General  Electric  Company. 

The  program  chairman  is  especially  indebted  to  Mr.  Cannon  of  the  Special 
Activities  Division,  Directorate  of  Technical  Operations  for  his  invaluable 
assistance. 
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ABSTRACT 


The  proceedings  contain  a  collection  of  16  papers  presented  at  the  Optimum 
System  Synthesis  Conference  held  at  the  Aeronautical  Systems  Division  11  -  13 
September  19 6Z.  The  meeting  was  direci^d  toward  defining  the  present  position  of 
optimum  system  synthesis  and  determining  guides  for  future  research  in  both 
applications  and  theory.  Most  papers  are  concerned  with  various  aspects  of 
recent  applications  and  theoretical  developments  in  optimal  control  such  as 
steepest  descent  techniques,  suboptimal  controllers,  optimum  filtering,  and 
functional  analysis  techniques.  Some  earlier  results  are  also  discussed. 


PUBLICATION  REVIEW 

This  technical  documentary  report  is  a  collection  of  the  papers  presented 
at  the  Optimum  System  Synthesis  Conference  and  is  published  to  make  this 
information  more  widely  available. 


Chief,  Aerospace  Mechanics  Branch 
Fli^t  Control  Laboratory 
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INTRODUCTION 

Since  the  Optimum  System  Synthesis  Conference  was  a  limited-attendance 
meetii^,  which  was  not  meant  to  be  widely  publicized,  many  readers  will  be 
interested  in  some  background  information.  The  major  aim  of  the  conference 
was  to  bring  together  some  prominent  mathematicians  and  engineers  working  in 
the  various  phases  of  optimum  system  synthesis  to  discuss  both  what  has  been 
accomplished  and  what  needs  to  be  done  to  make  optimum  synthesis  a  practical 
reality.  Three  sessions  were  set  aside  for  the  presentation  of  papers; 
these  papers  constitute  this  volume. 

In  addition  to  the  symposium  sessions  there  was  a  panel  and  forum  discussion 
of  the  general  topic,  *'Is  optimum  sjmthesis  practical?",  and  an  evening  session 
devoted  to  Informal  discussions  by  small  groups.  Each  group  had  a  particular 
subject  to  discuss,  which  lasted  until  either  the  subject  or  the  group  was  ex¬ 
hausted. 


Admittedly,  the  question,  "Is  optimum  synthesis  practical?"  is  much  too 
ill-defined  to  admit  of  a  concise  reply.  However,  some  attempt  should  be  made 
to  summarize  the  prevailing  opinion  —  A  task  one  or  two  levels  less  difficult 
than  predicting  the  wave  form  of  white  noise.  Despite  the  clamor  for  control 
system  optimization  techniques,  it  appears  that  quite  often  the  "optimum"  is 
a  chimera,  a  situation  which  results  in  what  might  be  called  The  Great  Optim¬ 
ization  Paradox.  On  the  one  hand  we  find  an  ever-growing  bo^  of  theory  and 
technique  for  solving  variational  problems  of  one  sort  or  another.  On  the  other 
hand  we  have  the  problems  of  the  control  system  designer,  many  of  which  can¬ 
not  or,  perhaps,  should  not  be  cast  in  the  form  of  variational  problems.  Yet 
there  seems  to  be  a  vigorous  attempt  by  both  sides  to  get  together,  even  though  it 
is  not  entirely  clear  that  there  is  any  extensive  common  ground. 

The  major  difficulty  is,  apparently,  our  inability  to  clearly  and  unambig- 
ously  specify  control  requirements.  Often,  the  critical  factor  is  not  performance, 
but  cost,  weight,  or  reliability.  When  we  seek  to  "optimize"  a  control  system, 
we  do  not  have  a  precise  ’.vay  to  characterize  the  optimum.  How  much  perform¬ 
ance  are  we  willing  to  trade  for  a  given  amount  of  reliability?  Indeed,  how'  do 
we  measure  performance?  These  are  far  from  trivial  questions;  the  utility  of 
the  powerful  synthesis  techniques  depends  upon  meaningful  answers,  answers 
which  are  not  readily  available. 

Not  all  control  problems  are  so  ambiguous,  particularly  the  outer-loop  or 
path  problems  (usually  associated  with  guidance,  aUhoU^  guidance  and  control 
are  the  obverse  and  reverse  of  the  same  coin).  In  cases  where  there  is  a  clear, 
quantitative  requirement  (e.  g. ,  minimiun  time,  minimum  fuel,  minimum  ex¬ 
pected  miss  distance)  the  techniques  hold  a  great  deal  of  promise,  and  we  can 
fhid  examples  of  actual  optimizations.  K  is  when  we  misapply  the  syntheses  to 
ill-defined  problems  that  we  siifier  our  worst  failures. 
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Another  obstacle  to  practicality  is  the  complexity  of  the  optimal  control 
law,  often  referred  to  as  the  problem  of  orbiting  a  7090.  Quite  often  the  criterion 
functional  that  is  being  optimized  is  not  too  sensitive  to  changes  in  the  control 
law,  and  we  can  do  ^almost  as  well”  with  a  much  simpler  system.  The  only 
other  answer  seems  to  be  picominiaturization  of  computing  equipment,  or  similar 
equipment  breakthroughs. 

As  a  final  point,  the  panel-forum  session  evidenced  a  healthy  trend  towards 
finding  out  vAiat  can  and/or  cannot  be  done  with  optimum  synthesis  techniques. 
The  practitioner  is  now  attempting  to  apply  the  theory. 
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ABSTRACT 


The  optimization  techniques  of  Pontryagin  (maximum  principle)  and 
Bellman  (dynamic  programming)  are  evaluated  as  feedback  design  tools. 
The  first  part  of  the  paper  is  devoted  to  a  discussion  of  the  advantages  of 
optimal  closed- loop  (feedback)  operation.  The  second  part  of  the  paper  is 
devoted  to  an  analysis  of  the  feedback  nature  of  the  two  optimization  tech¬ 
niques  cited  and  an  analysis  of  the  current  computational  difficulties  in 
applying  these  techniques. 
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FEEDBACK  DESIGN'^AND  OPTIMAL  CONTROL  THEORY 
John  G.  Truxal  and  Peter  Dorato 
Polytechnic  Institute  of  Brooklyn 


INTRODUCTION 


Recently  a  great  deal  of  interest  has  been  generated  in  the  application  of 
modern  optimization  techniques  to  problems  in  control.  In  the  past  the  term 
'‘control**  has  been  associated  with  closed-^loop  (feedback)  systems.  Yet  the 
current  literature  on  optimal  control  theory  is  largely  devoted  to  open-loop 
systems.  There  appears  to  be  some  limitation  to  the  application  of  tlie modern 
optimization  techniques  to  the  feedback  design  problem.  The  purpose  of  this 
paper  is  to  evaluate  such  presently  available  optimization  techniques  as  the 
maximum  principle  and  dynamic  programming  as  tools  for  the  design  of  feed¬ 
back  systems.  To  simplify  the  discussion  the  plant,  object  being  controlled, 
is  assumed  to  have  dynamics  given  by  the  first  order  scalar  equation 


5c  =  f(x,u)  (1) 

where  x  represents  the  plant  output  and  u  the  control  input.  The  performance 
criterion  is  assumed  to  be  given  by 


5 


F(x,  u)dt 


(2) 


The  object  of  optimal  control  theory  is  to  determine  a  control  input  u  which 
causes  S  to  be  an  extremal.  When  the  optimal  control  law,  denoted  u®,  is 
obtained  as  a  function  of  the  initial  output,  i.  e. , 

u°  =  t]  (3) 

thm  system  is  said  to  be  operating  open- loop.  When  the  optimal  control  is  ob¬ 
tained  in  terms  of  the  current  value  of  output,  i.  e.  , 

u°  =  4'[x(t);t]  (4) 

the  system  is  said  to  be  operating  closed-loop. 

The  first  part  of  the  paper  is  devoted  to  a  discussion  of  the  advantages  of 
closed^ loop  operation.  The  second  part  is  a  study  of  the  present  status  of 
optinxization  theory  as  a  feedback  design  technique. 
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ADVANTAGES  OF  CLOSED- LOOP  CONTROL 


Classically  feedback  systems  have  been  employed  to  solve  the  problems  of 


(1)  Plant  dynamics  modification 

(2)  Sensitivity  reduction 


It  is  fairly  easy  to  demonstrate  in  linear  time -invariant  systems  that  feedback 
does  provide  a  convenient  means  of  improving  system  dynamics  and  reducing 
plant  parameter  sensitivity  (ref.  1).  In  nonlinear  systems  the  advantages  of 
feedback  are  not  as  easily  demonstrated,  at  least^not  theoretically.  If  the  system 
performance  is  measured  in  terms  of  relative  stability  then  such  techniques  as 
equivalent  linearization,  describing  functions,*  and  Lyapunov* s  second  method 
may  be  used  to  evaluate  the  effectiveness  Jpf  feedback  as  a  plant- dynamics  modi¬ 
fication  tool.  To  show  that  a  given  f^jedback  control  law,  say  u  =  (x),  reduces 

the  sensitivity  of  the  output  with«respect*t.p  a  plant  parameter  w„  one  might 
proceed  as  follows.  Express  the*  plant  dynamics  with  parameter  w  as 

X  =  f(x,  u;w)  (5) 

•  * 

•  *  * 

and  define  the  sensitivity  measure  as  •  /  ^ 


s  =■- 


.8  w 


(6) 


From  the  theory  of  differential  equations  with  parameters,  t!ie  function  s  is 
known  to  satisfy  the  equation  (ref.  2) 


s 


(7) 


with  the  initial  condition  s(t  )  =  0.  With  no  feedback,  interpreted  in  this  case 
to  mean  u  =  0,  obtain  a  solution  of  equation  (7).  Denote  this  solution  Sq.  Denote 
by  S£  the  solution  of  (7)  with  feedback  applied,  i.e.,  .with  u  =  The  sensi¬ 

tivity  Improvement  may  then  be  evaluated,  for  example,  froha  the  relative  values 
of  }sf  j  and  .  Note  that  equation  (7)  is  linear  and  time- varying  and 

require?^  solution  o?^e  original  nonlinear  equation  (5)  via  the  term  8f/  0x.  A 
discussion  of  computer  solutions  of  (5)  and  (7)  has  been  given  by  Miller  and 
Murrary  (ref.  3).  However  to  obtain  closed  form  solutions  of  (5)  and  (7)  is, 
except  for  very  special  cases,  almost  impossible.  Hence  the  question  of 
sensitivity  reduction  in  nonlinear  systems  via  feedback  is  still  sax  open  question# 
In  optimal  control  systems  the  feedback  is  applied  to  modify  the  plant 
dynamics  in  an  optimal  way,  hence  plant  dynamics  modification  is  automatically 
taken  into  account.  The  stability  of  optimal  closed- loop  systems  is  generally 
assured  by  the  properties  of  the  perfprniance  criterion  chosen(ref.  4).  The 
result  follows  from  the  fact  that  S^,  where  S  denotes  the  value  of  S  when 
optimal  feedback  is  employed,  serves  as  a  Lyaj>unov  fvuxction  for  the  optimal 
closed- loop  system.  Thxis  if  F(x,  u®);,  where  u  -  ^(x(t);t) ,  is  positive  definite, 
then  the  Hamilton- Jacobi  equations  of  the  optimal  path  guarantee  that  S  is 
negative  definite;  indeed  the  Hamilton- Jacobi  equation  requires  that 

^  =  -F(x.u°)  (8) 


t 

y 

m 

i 

*4 
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In  optimal  closed-loop  systems  then,  stability  does  not  play  the  dominant 
role  it  does  in  classical  feedback  theory.  One  clear  advantage  of  closed- loop 
control  is  that  the  current  state  of  the  output  is  used  to  generate  the  control 
input  u.  Thus  if  the  output  is  suddenly  displaced  at  time  t*  by  some  disturbance, 
closed- loop  operation  assures  optimal  operation  in  the  remainder  of  the  optimi¬ 
zation  interval  {t*,T),  assuming  no  further  disturbances.  A  similar  disturbance 
in  an  open- loop  system  destroys  the  optimality  of  operation  in  the  interval 
(t*,T)  since  the  control  input  is  preprogrammed  for  a  fixed  initial  output  x(t  ). 

A  compromise  between  closed-loop  and  open- loop  operation  can  be  achievecf^ 
with  the  use  of  a  periodically  updated  (with  respect  to  the  value  of  x(t  ))  open- 
loop  control  law.  This  approach  has  some  practical  advantages  whicR  are 
discussed  in  the  next  section.  However  beyond  the  point  just  demonstrated  it  is 
difficult  to  illustrate  further  advantages  of  closed- loop  optimal  control.  Al¬ 
though  a  sensitivity  analysis  similar  to  the  one  previously  indicated  can  be 
performed,  the  optimization  techniques  in  current  use  do  not  assize  that  feed¬ 
back  remits  in  a  lower  sensitivity  value  than  open- loop  operation  ,  in  spite  of 
the  fact  that  experimental  studies  seem  to  indicate  that  feedback  in  optimal 
systems  does  reduce  sensitivity.  Thus  the  sensitivity  problem  in  optimal  con¬ 
trol  systems  is  much  like  the  sensitivity  problem  in  classical  nonlinear  feedback 
systems,  very  little  can  be  said  theoretically  about  the  relative  merits  of  feed¬ 
back  as  a  means  of  reducing  sensitivity.  The  only  distinct  advantage  of  closed- 
loop  optimal  control  then  appears  to  be  in  terms  of  possible  output  disturbances  . 


OPTIMIZATION  THEORY  AS  A  FEEDBACK  DESIGN  TECHNIQUE 


As  is  well  known  the  optimad  closed-loop  control  law  may  be  obtained,  at 
least  in  theory,  from  Pontryagin*s  maximum  principle  in  the  following  steps 
(ref.  5). 

(1)  Determine  the  function  u  which  maximizes,  for  a  minimum  of  S,  the 
Hamaltonian 


H  =  pf(x,u)  -  FCx,,u)  (9) 

Note  that  the  maximizing  u,  denoted  u°,  is  a  function  of  both  x  and  p,  and 
hence  may  be  written 


u°(t)  =  ajp(t),  x(t)} 


(10) 


(2)  Solve  the  set  of  equations 


X  =  f  (x,  u) 


9F 


(lla) 

(lib) 


^Indeed  a  basic  requirement  for  optimization  is  complete  knowledge  of  the  plant 
dynamics,  in  the  deterministic  case,  and  complete  knowledge  of  the  pertinent 
statistics,  in  the  stochastic  case. 
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subject  to  the  boundary  conditions 

(12a) 
(12b) 


x(t  )  =  X 

o  o 

p(T)  =  0 


It  is  assumed  here  that  the  final  time  T  is  fixed  while  the  final  output  x(T)  is 
free.  Note  that  the  solution  for  p  is  a  function  of  the  initial  output  x(t  )  and 
therefore  may  be  written 


p(t)  =  P 

(3)  Substitute  expression  (13)  back  into  (10)  and  let  t  :=  t^  to  obtain 

u°(t^)  =  «  0  (^0  ’  ^  J 

If  now  one  allows  t  in  expression  (14)  to  take  on  arbitrary  values  in  the 
original  interval  o/^optimization  (t  ,  T  ),  the  closed-loop  control  law  (4)  is 
obtained. 

As  is  also  well  known,  the  basic  difficulty  in  applying  the  above  procedure  is 
the  problem  of  obtaining  a  solution  to  the  nonlinear  two- point  boundary  value 
problem  of  step  2.  Basically  what  is  required  to  obtain  the  closed-loop  control 
law  is  a  mapping  of  the  values  of  x(t  )  into  values  of  p(t  ).  This  suggests  tha- 
arbitrary  values  of  x(T)  be  chosen  and  then  that  equations  (11a)  and  (11b)  be  run 
backwards  a  length  of  time  T-t  at  which  time  the  values  of  x(t  )  and  p{t  )  are 
recorded  for  the  required  mapping.  Of  course  this  procedure  must  be  repeated 
until  all  the  appropriate  values  of  x(t  )  are  spanned  and  all  values  of  time  in  the 
interval  (t  ,  T)  are  exhausted.  The  computer  time  and  storage  required  for  such 
a  procedure  is  however  prohibitive,  for  most  realistic  problems.  A  current  ap¬ 
proach  to  the  problem  is  to  use  some  steepest  descent  technique  to  obtain  an 
open-loop  control  law  and  then  update  from  time  to  time  the  value  of  x(t  )  (ref.  6, 
7,8).  The  success  of  such  a  method  depends  on  how  quickly  the  particuPar 
steepest  descent  technique  used  converges  and  how  often  the  output  data  must  be 
updated.  Further  application  of  the  maximum  principle  to  feedback  design 
depends  largely  on  the  development  of  numerical  techniques  for  the  solution  of 
nonlinear  two- point  boundary  value  problems. 

In  some  special  cases  the  maximum  principle  does  yield  an  explicit  solution 
for  the  optimal  closed- loop  control  law.  A  notable  example  is  the  case  of  a 
linear  plant  with  unconstrained  input  and  integral-quadratic  performance 
criterion  (ref.  9,10). 

Dynamic  programming  is  a  feedback  design  technique  by  its  very  nature.  ^ 
Unfortunately,  however,  dynamic  programming  also  suffers  from  some  basic 
computation^  difficulties.  To  illustrate  the  feedback  nature  of  dynamic  pro¬ 
gramming  and  also  its  computational  difficulties  the  technique  is  briefly  outlined. 

^  Of  course  in  the  determinstic  case  any  feedback  solution  can  always  be  corii- 
verted,  via  the  plant  dynamics,  into  an  open- loop  solution.  The  basic  feedback 
nature  of  dynamic  programming  is  inescapable,  however,  in  the  study  of  stoch¬ 
astic  systems.  See,  for  example, Bellman  (ref.  11),  sec.  10.4. 
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First  Assume  that  the  plant  dynamics  and  performance  criterion  have  been 
made  discrete  in  tirne^  thus  consider 

^+1  = 


s  -  2  °<V  \) 


where,  for  convenience,  the  notation  x(kAt)  =  is  used.  Denote  by  Sj^  the 
value  of  S  which  results  from  an  optimal  input  policy  over  the  interval  N) 
and  note  that  S?  is  a  function  only  of  hence  may  be  written 
optimal  control  law  is  then  determined  in  the  following  steps. 

(1)  Start  at  the  end  of  the  process  and  search  for  a  value  of  u^  which  mini¬ 
mizes,  assuming  one  wants  to  minimize  S,  the  function  G(Xj^,  u^^.  Note  that 
Uj^  is  a  function  of  Xj^.  Let 


Un  = 


and  evaluate 


(2)  P^ceed  one  step  backwards  and  minimize,  with  respect  to  \x^  the 
function  £g(Xj^_j,  Uj^  j)  +  S°  •  where  Xj^  =  g(Xj^_j,  The~results 

of  this  step  then  yieias 

’"N-1  =  ‘‘'N-1 

and 

®N-1  ^  r°^^N-l’  “N-1^  ■'■^n6^^N-1’^N-iS 

Uik.T’  1 


(3)  Proceed  backwards,  iterating  the  above  procedure,  until  the  entire 
interv^  (1,  N)  is  exhausted,  to  obtain  the  closed- loop  control  law 

“k  "  ’  l<k<  N  (21) 
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Note  that  the  minimization  technique  just  described  requires,  at  any  stage 
k,  a  search  for  Uv  in  terms  of  all  possible  values  of  x,  and  storage,  for  use  in 
the  next  stage,  oi  the  function  Sj^{x«  ).  In  addition  all  me  functions  )  must 

atlso  be  stored*  For  practi  cal  problems  the  amount  of  storage  required 
exhausts  the  capacity  of  most  modern  day  computers.  The  application  of 
dynamic  programming  as  a  feedback  design  technique  then  seems  to  depend 
critically  on  the  development  of  methods  which  reduce  the  amount  of  storage 
required  and/or  the  design  of  computers  with  larger  storage  capacity. 

Even  if  the  computational  difficulties  of  the  maximum  principle  and  dynamic 
programming  are  solved,  there  still  remains  the  problem  of  implementing  the 
closed- loop  control  law  with  currently  available  hardware..  In  this  connection 
it  should  be  noted  that  the  control  law  is  generally  a  nonlinear  function  of  the 
state  of  the  plant.  Here  two  separate  problems  arise:  one  is  the  problem  of 
measuring  the  state  of  the  plant,  which  often  requires  the  measurement  of  high 
order  derivatives,  and  the  other  is  the  problern  of  generating  the  required 
nonlinear  function  to  the  plant  state.  If  a  computer  (digital)  is  used  as  the 
controller  element  in  the  feedback  loop,  then  the  computer  must  accomodate 
the  storage  of  the  control  daw  for  all  possible  values  of  x,  and  for  all  k 

in  the  range  (1,  N).  One  possible  solution  to  this  aspect  of  the  problem  is  the 
use  of  "approximations  in  policy  space"  which  yield  simpler  control  laws,  at 
the  price  of  suboptimal  operation  (ref.  11,12). 

In  spite  of  the  difficulties  cited  above,  optimal  control  theory  does  provide 
an  organized  approach  to  the  design  of  complex  feedback  systems.  The  compu¬ 
tational  difficulties  encountered  should,  in  fairness,  be  weighted  against  the 
rather  sophisticated  nature  of  the  problem  considered.  Indeed  in  many 
applications, especially  nonlinear  stochastic  systems,  the  alternative  to  optimal 
control  theory  is  no  theory  at  all. 
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I.  INTRODUCTION 


Much  of  the  current  work  on  control  systems  optimization  has  centered 
on  the  establishment  of  necessary  conditions  that  the  optimum  solution  must 
Satisfy.  In  many  cases  these  conditions  take  the  form  of  a  functional  equation 
or  a  vector  partial  differential  equation  that  the  solution  must  satisfy.  Much 
less  volume  of  work  has  been  devoted  to  the  equally  important  problem  of  ob¬ 
taining  efficient  computational  algorithms  based  on  successive  iteration 
schemes  which  converge  to  the  optimal  solution.  [1,  2,  3.  4].  ^his  paper  deals 
with  such  a  procedure  introduced  in  [3]  for  a  class  of  dc  Lermin\istic  control 
problems  using  methods  of  abstract  functional  analysio  which  afford  a  succinct 
and  general  solution  in  many  cases  and  offer  a  point  of  departure  for  problems 
where  any  systematic  analysis  of  a  general  enough  nature  wouldj  appear  to  be 
difficult. 


The  computational  method  employed  here  is  the  method  of  steepest  de¬ 
scent  in  Hilbert  space,  an  early  version  of  which  is  due  to  Kantorovich  [5]. 

In  setting  the  control  problem  in  the  function  space  format,,  we  make  use,  for 
instance,  the  Theory  of  (Frechet)  analytic  functions  over  abstract  spaces, 
specialized  to  Hilbert  space  [6]. 

In  Section  II,  we  consider  the  general  problem  for  linear  systems  and/^ 
show  how  it  can  be  set  entirely  as  a  function-space  problem  leading 
general  results.  In  Section  IK,  we  specialize  to  linear  dynamic  ex¬ 

ploiting  the  compactness  of  the  operation  involved!  The  coJ^P^tati^Pixiethod 
is  explained  in  Section  IV.  The  specialization  to  the  final  value^Q^j^em  for 
linear  systems  is  contained  in  Section  V.  In  Section  VI,  to  non¬ 
linear  systems  are  indicated,  and  finally  in  Section  VII,  of  some  com¬ 

puter  studies  based  on  the  proposed  method  are  given.y/^ 


II.  FORMUI^rj.jQj^T  yor  linear  SYSTEMS 

We  begin  by  cons.idej^^g  ^  linear  system.,  since  it  is  important  in  its 
own  right  and,  moreove^^  provides  a  point  of  departure  for  the  extension  to 
nonlinear  systems  in  Section  VI,  Let  x(t)  represent  the  output  vector 

of  a  linear  system  result  of  the  applied  control  vector  u(t).  Then  x(t) 
will  be  related  f^^iG\onally  to  u(t)  often,  though  not  necessarily,  by  means 
of  a  differential  Let  x^j(t)  represent  the  desired  response.  The 

optimization  proVi^j.^^  ,2s  one  of  minimizing  some  functional  of  the  difference 


eCt)  =  x(t\. 


"^^^Xriterion  chosen  in  this  paper  is  the  integral  of  the  squared  magni¬ 
tude  some  finite  m'erval  which  we  can  normalize  to  [0,  T].  Thus  we 
wasj>4^  determine  u(t)  which  minimizes 


J  II  e  (t)  I  j  ^  p  (t)  dt  ;  II  e  (t)  [j  ^  =  e  (t)  e  (t) 


Here  p(t)  is  a  non-negative  wei^^ht  function  and  the  asterisk  denotes  the  ad¬ 
joint.  To  avoid  the  language  of  6 -functions,  we  assume  more  generally  that 
we  ha^;e  a  finite  measure  m^  over  the  Borel  sets  of  [0,  T]  and  we  wish  to 
minimize 


e  (t)  II  ^  dm^ 


where  ir  denotes  the  interval  [0,  T],  Here  e(t)  will  be  continuous  by  virtue  of 
x(t)  and  x^(t)  be  mg  continuous.  Usually,  there  will,  in  addition,  be  some  con¬ 
straint  on  the  control  vector  such  as 


J  j!  u  (t)  II  ^  dm^  = 

it 

where  m2  is  another  finite  measure  on  the  Borel  sets  of  ir ,  and  again  u(t)  will 
be  Borel-measurable.  By  taking  the  measure  mi  to  be  a  jump  at  T,  we 
specialize  to  the  "final  value  problem".  But  we  shall  see  that  this  class  of 
problems  can  be  solved  constructively  with  no  restriction  on  m^  and  m2, 
other  than  finiteness.  For  this  it  is  convenient  to  introduce  the  real  Hilbert 
space  (actually  L2  space)  Hj  of  n-dimensional  functions  square  integrable  over 
JT  with  respect  to  measure  m^  and  H2,  the  L2  space  of  m-dimensional 
functions  square  integrable  over  t  with  respect  to  measure  m2-  Then  x(t), 

0  ^  t  S  T,  will  be  in  Hj  and  the  control  vectors  can  be  taken  in  H2.  Moreover, 
for  a  linear  system,  we  have 
T 

x(t)  =  f  W(t,  a)  u  (a)  da  (1) 

‘^b 
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Of  course,  for  physical  realizability,  unessential  restriction  for  our  problem 
is 


x(t) 


W(t,  ct)  u  (a)  da 


(2) 


Such  a  relation  may  come  from  a  differential  equation  but  not  necessarily  so. 
Now,  assuming  W(t,a)  to  be  continuous  for  instance,  (1)  defines  a  linear 
bounded  operator  from  H2  into  Hj.  We  denote  this  operator  by  L,  so  that  we 
represent  (1)  by 


L  u  =  x;  u  e  Hg,  x  e 
In  this  frame  then,  we  want  to  minimize 


I  Lu  -  g 


I: 


The  subscripts  1  and  2  denote  the  norms  in  and  H2  respectively.  is 
a  given  element  in  and  minimization  is  over  a  sphere  of  radius  M  in  H2, 
or  more  generally  it  can  be  over  any  closed  convex  set  (with  non-empty  in¬ 
terior  usually).  The  minimization  can  be  made  slightly  more  general  by 
adding  another  perturbing  term  and  considering 


C(u) 


U'Lu  -  g||^  + 

1 


Ku,  u 


2 


(3) 


where  K  is  a  linear  bounded  non-negative  operator  on  H2  into  H2.  For  ex¬ 
ample,  it  may  be  multiplication  by  a  non-negative  number  corresponding  to 
a  ** Lagrange  multiplier'*,  or  more  generally  it  may  be  multiplication  by  a 
positive -definite  matrix  K(t).  Let  L*  be  the  adjoint  of  L,  then  L*  maps 
and  H2.  We  can  then  rewrite  C(u)  as 

C(u)  = 

Now  L  L  is 


-  2 

u,  L*gi  +  r  g,  g 

■^2 

L  J2  L  J 

(L  L  +  K)  u,  u 
Iso  non-negative,  so  that  so  is  the  sum 


L*L  t  K  =  R 


(4) 


Assume  that  K  is  such  that 


Ku  ,  u 


whenever  L  Lu 


0 


^More  generally,  we  C|n  consider  the  minimization  of 
P(Lu-g),  Lu-g 

where  P  is  a  linear  bounded  positive  definite  operator  on  into  Hj. 
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Then  R  has  a  bounded  Linear  inverse.  Let 


R 


— 

u  = 
o  L 


L  L  +  K 


Uq  =  L  g 


(5) 


That  u  is  then  the  unique  optimal  minimizing  solution  is  apparent  from 
[see  Reference  4] 


C(u)  =  Cfu  1  + 
o 


R(u-u  ),  u  -  u 
o  o 


(6) 


and  the  second  term  is  non-negative.  This  establish^.,  C(u  )  ac  the  unique 
absolute  minimum,  whereas  the  fact  that  it  is  a  local  ml.iimum  may  be 
shown  in  various  ways.  Moreover,  the  optimal  solution  u^  is  of  the  form 
by  using  (5) 

Ku  =  L  g-Lu 
o  L  O  J 


or 


u 

o 


-1  ' 

(L  K 


g  -  L  u 


o 


If  K  is  just  a  positive  multiplication  k,  this  reduces  to 

*  ig  -  L 

u  =  L  - 5 - ^ 

o  k 

so  that  u^  itself  is  in  the  range  of  L  .  Hence  using  this  and  setting 


♦ 

u  =  L  y 
o 

in  (5),  we  obtain  an  equation  in  H^. 
LL,  y  +  ky  =  g 


(7) 


This  yields  a  simplification  if  is  finite  dimensional,  as  would  be  the  case 
if  m^^  has  a  jump  at  T. 

The  main  reason  for  introducing  K  is  the  application  to  the  constraint 
problem:  Suppose  the  convex  set  A  is  the  class  of  all  u  such  that 

II  u  II:  2  s  M 


Then  we  have  the  following  theorem:  There  is  a  non -increasing  sequence  of 
positive  number  such  that 
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-1 


L*L  +  k  I 
n 


♦ 

L  g 


u 

n 


I  being  the  identity  operator,  where  u  e  A  and 

lim  II  L  u  -  g  |j  ^  =  Inf  ||  L  u  -  gf|  ^ 
n  "  ueA 


(8) 


The  proof  of  this  theorem  uses  (6)  and  may  be  found  in  [4].  Here  we  shall 
sketch  it  for  completeness.  Suppose  there  is  an  element  u  in  A  such  that 


L*L  u  *  L*  g  (9) 


Then  it  is  clear  that  we  can  find  a  decreasing  sequence  of  positive  numbers 
kjj  such  that 


u 

n 


L*L  +  k  I 
n 


♦ 

L  g 


belongs  to  A.  Since  for  each  n, 

L*L  +  k  I 
L  n  J 

has  a  bounded  inverse,  then 

II  u^  ll  S  ||[L*L  +  k^  I]’^  L*L  II  ||u|  |  S  II  u  || 
Thus  using  (6)  we  have 


C(u)  =  C(u  )  +  k 


n 


u  ,  u  ”  u 


nun 


n  J 


and  from  which  it  follows  that,  as  k 


n 


C(u^)  -  C(u) 


as  required, 
find  positive  k 

[  L*L  +  k 


If  there  is  no  u  in  A  such  that  (9)  holds,  then  we  can  clearly 

such  that 
o 


I 

o 


L  g  =  u 


and 
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It  may  be  shown  as  in  [4]  that  this  element  yields  the  unique  minimum,  since 
for  any  k  < 


L*L  +  k 


1-1 


❖ 

L  g 


II  >  M 


and  hence  is  not  in  A.  In  particular,  we  note  that  the  optimal  element  is  al¬ 
ways  in  the  range  of  L’*',  or  possibly  its  closure. 


III.  THE  MINIMIZATION  PROBLEM  FOR  LINEAR  DYNAMIC  SYSTEMS 

In  this  section,  we  shall  specialize  our  discussion  to  the  minimization 
problem  for  a  linear  dynamic  system.  We  shall  take  m^  and  m2  to  be 
Lebesgue  measure.  Suppose  now  that  the  dynamic  system  is  described  by 
a  differential  equation: 

x(t)  =  A(t)  x(t)  +  B(t)  u{t) 


where  A(t)  is  an  n  x  n  matrix  and  B(t)  is  an  n  x  m  matrix.  We  shall  have 


x(t) 


with 


Xo(t) 


+ 


T)  u(T)  dT 


(10) 


W(t,T)  =0  forT>t 


Here  W(t,  T)  is  an  n  x  m  weighting  function  matrix  of  the  system  and  XqU)  is 
the  output  due  to  initial  condition.  We  can  now  define  a  linear  operation  L 
from  H2  into  as 


v{t) 


W(t.t)  u(T)  dT 


L  u 


V 


0  ttST 

The  main  simplification  that  we  shall  exploit  in  this  section  is  the  com¬ 
pactness  of  the  operator  L  by  virtue  of 


T^T 


J  J  |[w(t.T)[|  dtdT 


<00 


'O  '"O 


Here  the  adjoint  L*  mapping  into 
T 

h(T)  =  f  W*{t,  T)  g(t)  dt;  L*g  »  h 
0  T  S  T 
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Now 


C(u) 


L  u  -  g| 


Lu,  Lu 


-  2 


* 

L  Lu,  u 


g.  Lu 


+ 


* 

L  g,  u 


1 


where 

g(t)  =  x^(t)  - 


Denoting  L  L  by  R,  we  note  that  R  is  non-negative,  self-adjoint  and  an  in¬ 
tegral  operator  with  kernel 


G(T 


1' 


W*(t, 


Ti)  W  (t,  T^) 


dt 


It  is  also  compact  and  thus  is  characterized  by  a  countable  number  of  non¬ 
negative  eigenvalues.  Moreover,  for  any  u  in  Hg,  we  have 


R  u 


00 


where  {X^^}  are  the  set  of  non-zero  eigenvalues  and  are  thc3  set  of  corres¬ 
ponding  orthonormalized  eigenfunctions,  and  {X^}  are  arranged  to  be  monotone 
nonincreasing. 


We  can  now  phrase  the  minimization  problem  entirely  in  terms  of  the 
eigenfimctions.  For  this  we  note  that  for  any  u  in  H2,  we  have  the  unique 
orthogonal  decomposition 


u  -  I 
1»1 


where 


u,  6. 


L  u  ,  L  u 
o  o 


»  0 


(11) 


Similarly,  since  {L^^^}  are  orthogonal,  we 


have  the  orthogonal  decomposition 
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*  [  g.  1 

g  “  2 


i«i 


L  +  g 

®c 


^  So*®'  S 

i  =  l 


[g.  L  0.]- 


X. 

1 


<  00 


Hence  we  have 

OO 


C(u)  =  ■  2  a  [g.  L^J  +  l|gH 

1=1  ^  ^  1  1 
and  completing  the  square  in  each  term 


1=1  1 


[g-MJ  ,2  [g.  Mj] 


X^i 


+  llg 


“  [g.  L^J  2  2 

=  -Z  HIgll  =  llg  J[ 


i»l 


X. 

1 


Hence  it  is  clear  that  if  we  set 


i  = 

We  have  that 


II  [S.  L  «.] 


,  ,  X. 

1  =  1  1 


lim  C(u  )  =  g  =  Inf  C(u) 
n  n  oq  • « 

n  u 


and 


iim  R  u  =  L  g 
n  ® 
n 


A  necessary  and  sufficient  condition  for  an  element  h  in  H- 


Inf  C(u)  =  C(h) 
n 

is  then  of  course  that 
«  [g.  L 
Li 


<00 


1-1  X 


i 


such  that 


18 


and  in  that  case 

S  (  g.  L  ♦,  I 

h  -  I  - j-i- 

.  i  »  1  i 

Suppose  now,  in  addition, 

InfC(u)  =  0 
u 


Then  we  notice  that  g  * 

®o 


;  Rh  *  L* g 
that  h  is  such  that 

0  and 


L  h  =  g 


and  what  is  more  important,  for  any  u  such  that 


L  u  =  g 
we  have 

u  -  h,  0.  J  =  0  for  every  i 

and 

llulUlIhll 

by  virtue  of  the  orthogonal  decomposition  (11).  If  we  want 

Inf  C(u)  =  0 
u 

for  every  g  in  then  of  course  we  must  have  that  {l  is  complete  in  Hj 
On  the  other  hand,  the  more  common  problem  is  one  which  constraints  u  to 
be  in  some  bounded  convex  set.  Suppose  we  enclose  this  set  in  a  sphere  of 
radium  M  and  consider,  instead,  u  such  that 


First  of  all,  if  we  take  a  k>  0  and  minimize 
C(u)  =  II  L  u  -  gll  j  +  k  ||u||| 

S' 

we  have  in  terms  of  the  decomposition  ( 1 1 ) 


(12) 


C(u)  =  Yj  + 
i=t 


[  g,  L,^.]  2  [S> 


2n  . 


^i  ■  X.+  k  ) 


(X.  +  k)  ‘ 


llg|| 


Hence  setting 

[g.  L^il 

'  X.+  k 

1 


u  =  0 
o 


We  note  that 

“  [g.  L0  ] 

|R  +  kl  u  =(R  +k)  T  — -  ‘ 

1=1  ■'  ' 
OO 

"  Z  [Si  L^.j  ^.  =  L  g 

i=l  ^ 

or 

u  =  [R  +  k]"^  L*  g 


To  answer  the  constraint  problem,  we  may  look  upon  (12)  as  the  usual  per¬ 
turbation  using  a  Lagrange  multiplier,  and  hence  all  we  have  to  do  is  to  use 
the  smallest  k  so  that 

E  - ^2  =  ^ 

i=i  (\+kr 


unless  of  course 


[giL^il' 


< 


M 


1 

That  this  does  yield  the  absolute  minimum  has  already  been  shown  generally  in 
Section  II. 


The  derivation  given  above  for  obtaining  the  optinial  control  depends  on 
the  knowledge  of  the  eigenvalues  and  eigenfimctions  of  the  operator  R.  Since 
they  are,  in  general,  infinite  in  number,  this  approach  is  not  suitable  for 
solving  the  synthesis  problems.  In  the  next  section,  we  shall  present  a 
method  of  steepest  descent  in  Hilbert  space,  which  will  overcome  these 
difficulties. 


IV.  THE  METHOD  OF  STEEPEST  DESCENT  FOR  MINIMIZING  A 
QUADRATIC  FUNCTIONAL 

In  this  section  we  shall  indicate  a  computational  method  based  on  the 
steepest  descent  method  in  Hilbert  space  for  solving  the  minimization  prob¬ 
lem  associated  with  a  quadratic  fimctional.  An  early  exposition  of  such  a 
method  is  due  to  Kantorovich  [5],  The  essential  idea  of  this  niethod  is  con¬ 
tained  in  the  following:  In  seeking  the  minimum  of  a  quadratic  functional 
Q(u),  an  arbitrary  initial  guess  Uq  is  assumed.  We  obtain  the  gradient  at  this 
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point,  i,  e. ,  finding  an  element  z  such  that  ^  ^  Q(Uq  +  ez)^  will  be  maximized 

at  e  -  0.  Let  be  such  an  element.  Since  Q(uo  +  e  Zq)  is  a  second  degree 
polynomial  of  e,  it  will  attain  a  minimum  for  some  e  .  Then  the  element 
^1  “  ^0^0  adopted  as  the  next  approximation  and  the  whole  pro- 

cedure  Ccin  be  repeated  as  many  times  as  accuracy  required. 

A  geometrical  interpretation  of  this  method  is  fairly  clear.  In  the  space 
where  Q(u)  is  defined,  the  surfaces  Q(u)  -  C  are,  in  general,  a  family  of 
similar  ellipsoids  with  center  at  the  minimum  point.  The  initial  approxi¬ 
mation  Uq  shall  lie  on  a  certain  ellipsoid  of  the  family.  From  this  initial  point 
we  shall  move  along  the  direction  of  the  gradient  at  this  point,  i.  e. ,  along  the 
normal  to  this  ellipsoid.  We  shall  reach  a  point  Uj  where  the  value  of  Q(u)  is 
the  least  on  this  normal,  i.  e. ,  a  point  where  the  normal  line  is  tangent  to 
some  ellipsoid  of  the  family.  From  this  point  ui,  we  shall  go  along  the  new 
normal  to  this  ellipsoid  Q(u)  =  QCuj),  and  the  whole  process  is  then  repeated. 


The  quadratic  functional  to  be  minimized  has  the  general  form 
Q(u)  =  Tu,  u  -2  h,u 


(13) 


where  T  is  a  positive  definite  or  a  non-negative  definite  operator  in  Ho.  Let 
us  now  substitute  u  by  Uq  +  ez.  Then  ^ 


Q(u,,  +  ez)  =  Q(u  )  +  2e 
o  o 


T  u  -  h,  z 

Tz,  z 

tv  V  . 

2 

- 

(14) 


In  seeking  the  direction  of  the  gradient,  it  is  necessary  to  find  z  such  that 

+  =2[tu  -h,z 

e=o  2 

is  maximized.  According  to  Schwarz's  inequality,  it  will  be  achieved  if 


z  =  T  u  -  h 
o  o 


(15) 


Here  R  u  -  L  g  shall  be  referred  to  as  the  gradient  of  the  fimctional  Q.  With 
the  choice  of  z^  as  given  by  (15),  (14)  will  attain  its  minimum  if 

II  z  ,  "  ' 

c  =  - 


“o  "2 


o  [T  z  ,  z  1 
o  o\ 


(16) 


By  repeating  the  above  procedure,  it  is  evident  that  at  the  nth  step,  we  have 
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(17a) 


u 

n 


u  ,  +  €  , 

n-1  n-1 


(T  u 


h-I 


-  h) 


with 


Vl”2 


‘n-l  [Tz 


(17b) 


z  ,  =  T  u  -  h 
n-1  n-1 


(i7c) 


We  note  that  we  improve  performance  at  each  step,  that  is  to  say: 

lUnllj 

n+1  n  [Tz^,z^] 


(18) 


It  can  be  shown  [3]  that  Q(u  )  decreases  to  the  optimum: 

n 


lim  Q(u  )  =  Inf  Q(u) 
n  u 

Many  variations  on  this  basic  idea  are  possible.  However,  the  method 
has  to  be  substantially  modified  when  the  control  u  is  constrained,  such  as  u 
is  restricted  to  some  closed  bounded  convex  set  with  the  origin  as  an  interior 
point.  In  this  case,  we  can  try  adjusting  e  and  to  satisfy  the  constraint 
in  many  ways  but  the  convergence  of  the  sequence  to  the  optimum  would 
appear  to  be  difficult  to  obtain  in  general.  By  slightly  enlarging  the  convex 
set  to  be  the  smallest  sphere  containing  it,  we  can  obtain  a  general  con-' 
vergence  proof  for  a  modified  steepest  descent  method,  as  shown  in  [4].  Let 
us  assume  that  the  radius  of  the  sphere  is  M.  Then  we  have  seen  that  the 
optimal  solution  is  given  by 

u  *  (R  +  k)"^  L*g 
for  some  k  such  at  ||  u  |j  ^  =  M . 

Let  us  assume  that  at  the  nth  iteration,  we  have 

II  u  f 

^  2 


We  shall  now  consider. 


u  ^^(k)  =  u  -  e  z 
n+1  n  n  n 


as  a  function  of  positive  k. 


where 
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|2 


n' 


e  = 
n 


[(R+k)z  ,z  1 
n  n  , 


♦ 

z  =  (R+k)  u  -  L  g 
n  n 


Now 


u 


h+1 


(k)|l 


2 

2 


is  a  continuous  function  of  k  and  approaches  to  zero  as  k  goes  to  infinity. 
Suppose 

2  „ 

II  u^^^(O)  II  5  M 
2 

Then  we  define 


u  =  u  .,(0) 
n+1  n+1 


Otherwise,  let  k  be  such  that 
n 


“n+l'V  11/ “ 


and  define 


u  =  u  (k  ) 
n+ 1  n+ 1  n 


It  can  then  be  shown  that 


lim  Q(u  )  =  Inf  Q(u) 
n 

n  u 

for  II  u  II 2  =  It  must  be  noted  here  that  the  sequence  is  not  necessarily 
monotone  at  the  early  stage  of  the  iteration. 

In  practice,  we  are  not  really  interested  to  haye  ||  u  ||  exactly  equal  to  M. 
A  more  realistic  constraint  for  u  is  that  for  some  small  ^  0,  either 


2 


or 
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Thus  we  shall  define 


Vi  '  Vi*"’ 


if 


u  ^  (0)  5  M 

n+1 


and 


Vl  ' 


if 


-  6  S  ilu^^^(k^)  I  £ 
2 


This  is  actually  what  we  would  do  when  we  carry  out  the  computation  on  a 
digital  computer. 

V.  THE  FINAL-VALUE  PROBLEM  FOR  LINEAR  SYSTEMS 

In  solving  the  general  control  problem,  the  steepest  descent  method  can 
be  used  to  obtain  the  optimal  solution.  However,  when  we  are  only  inter¬ 
ested  in  controlling  the  final  output  vector,  then  there  is  a  reduction  in  dimen¬ 
sionality  affording  a  corresponding  simplification.  In  fact,  the  range  of  L  be¬ 
comes  finite  dimensional  so  that  matrix  methods  suffice. 

More  specifically,  we  want  to  minimize 

T 

H  X(T)  -  gll^  ;  x(T)  =  f  W(T.t)  u(t)  dt 

where  now  g  is  a  finite  dimensional  vector.  In  our  general  treatment,  we 
have  only  then  to  take  m^  to  be  a  jutnp  at  T.  Hence  is  finite  dimensional. 
For  simplicity  let  us  take  m2  to  be  Lebesgue  measure.  Then  again  L  is  com~  . 
pact,  and  moreover 

L*g  =  W(T,t)*  g 

so  that,,  using  (7)  we  know  that  the  optimal  solution  is  of  the  form 
* 

u  =  L  y 

Also  we  need  to  consider 


* 

♦  ♦ 

* 

L  L  +  kl 

L  y  =  L 

(LL  +  kl)  y 

♦ 

=  L  g 


and  now 
♦ 

LL 

being  an  operator  on  a  finite  dimensional  space  must  be  a  square  matrix,  and 
indeed  it  is  the  n  x  n  matrix 

♦  * 

A(T)  =  LL  =  \  W(T,  T)  W(T,  T)  dT 

‘^o 


and  the  optimal  vector  y  is  given  by 


A(T)  +  k 


y  =  g 


for  some  k.  The  constr  aint 


H  ull 2 

now  becomes 
y*  A(T)  y  £ 


(19) 


The  steepest  descent  can  then  proceed  using  (17a)  and  (17b),  so  that  for 
instance 


'n+1 


=  y  +  € 


„[( 


A(T)  +  k 


n) 


On  the  other  hand,  we  can  also  use  the  eigenfunction  approach,  the  eigen¬ 
functions  of  R  being  finite  in  nuniber,  since  range  of  R  is  finite  dimensional. 

In  particular  we  note  from  Section  III  that  for  the  minimum  to  be  zero  for  every 
g,  we  must  have  that  R  must  have  exactly  n  eigenfunctions  corresponding  to 
non-zero  eigenvalues.  It  should  be  noticed  that  the  eigenvalues  for  R  is  exactly 
the  eigenvalues  for  A(T),  We  can,  of  course,  readily  translate  (19)  in  terms 
of  the  eigenvectors  of  A(T).  Let  /3^  be  the  orthonormalized  eigenvectors  cor¬ 
responding  to  the  non-zero  eigenvalues  of  A(T).  Then^the  optimal  y  is 
given  by 


[g.^ii 

X.+  k 


It  may  also  be  noted  that  a  necessary  and  sufficient  condition  fbr  A(T)  have  a 
zero  eigenvalue  is  that 
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(20) 


W(T.  t)*  y  =  0  for  0  2  t  S  T 

for  some  non-zero  y,  since 

*  II  *  11^ 

y  A(T)  y  =  \  ||  W(T.  t)  y  ||  dt  =  0 

*^0 

For  dynamical  systems  characterized  by  differential  equations,  (20)  can  of 
course  be  carried  further. 

VI.  NONLINEAR  SYSTEMS 


In  this  section,  we  shall  indicate  some  extensions  of  the  previous  theory 
to  nonlinear  systems.  We  shall  not  strive  for  maximum  generality  here,  al¬ 
though  much  of  the  theory  is  capable  of  further  generalization.  A  key  point 
in  dealing  with  nonlinear  systems  is  the  specification  of  the  nonlinear  system. 
We  may  consider  a  dynamic  system  characterized  by  a  nonlinear  differential 
equation 

X  =  F(x,u,t)  (21) 


where  x(t)  is  the  system  state  vector  and  u(t)  is  the  control  vector.  We  want 
to  minimize,  as  before 


C(u) 


2 

r  ||Ax(t)  -  g(t)  II  d  +  k  y  II  u(t)  || 
Sr  Sr 


d  m. 


(22) 


where  ||  ||  denotes  the  Euclidean  norm.  Assuming  there  is  a  unique  so-r 

lution  to  the  Equation  (21)  for  fixed  initial  conditions,  we  have  then  a  mapping 
from  H2  into  as  before,  except  that  now  in 


A  X  »  AN(u) 


N  is  not  a  linear  operator  but  rather  a  nonlinear  operator  or  function.  We 
shall  assume  that  F(. ,  . )  is  such  that  N(u)  is  analytic --that  is,  locally  bounded 
and  G- differentiable.  Then  we  wish  to  minimize 

2  2 

C(u)  =  ||  an  (u)  -  g  II  +  k  |u  ||  (23) 

1  2 

From  this  point  of  view  then,  all  we  need  is  N(u),  and  it  does  not  matter 
whether  it  comes  from  a  dynamic  system  or  npt.  It  is  clear  that  (23)  is  also 
analytic,  Moreover,  the  Frechet  derivative  is  (in  the  notation  of  Reference  6) 
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6  1^  C(u);  h  J  =  2  A6  [  N(u);h  ]  .  AN(u)  j 

*  ]‘ 


-  2  A6  [  N(u);h  ] 
+  2k  u,  h  j 


(24) 


But,  since  6[C(u);  h]  is  a  continuous  linear  homogeneous  polynomial,  we  must 
have 


6  N(u);  h 

=  Lh 

«  r  C(u);  h 

.  [  V.  h] 

L 

2 

where,  now,  from  (24),  we  note  that 

v  =  2  [  AL 

* 

A  N(u)  -  2 

AL 

g  +  2  k  u 


(25) 


and  moreover,  the  derivative  zero  leading  to  an  extremum  for  C  such  that 


0  =  V 


"1*  r  1* 

^  AL  AN(u)  -  AL 


g  +  k  u 


or 


k  u  =  AL  j  g  -  AN(u) 

This  result  generalizes  (7).  Substituting 
u  =  (AL)*  y 

in  (26),  we  again  obtain  for  y  the  equation  in 

AN  (AL)*  y  J  +  k  y  =  g 


(26) 


(26a) 


We  note  that  (26)  and  (26a)  are  ho  longer  linear  equations,  although  if  is 
finite  dimensional,  (26a)  is  of  course  also  a  finite  dimensional  vector  equation. 
On  the  other  hand,  steepest  descent  method  in  this  context  becomes: 


u 


h+1 


u  +  e  z 
n  n  n 


(27) 
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where 


z 

n 


-  AL  g  +  k  u 
^  J  n 


and  €  is  chosen  to  minimize 
n 


Although  we  can  thus  inprove  the  approximation  at  every  stage,  convergence 
to  the  optimum  is,  in  general,  much  harder  to  prove. 

The  main  calculation  is  the  derivative  6[N(u);h].  We  shall  now  examine 
this  in  some  special  cases.  First  of  all,  being  analytic,  we  know  that  we  can 
expand  N(u)  as  a  power  series  which  can  then  be  approximated  by  a  polynomial 
of  degree  m  say.  For  example,  suppose  that  u(t)  is  one  dimensional,  and  we 
take  m  =  2 : 


x(t)  =  J  Wj(t,  s)  u(s)  ds  +  ^  ^ 


o  o 


W^Ct,  S2)  u(s  j^)u(s2)dSj  ds^  (28) 


then 


6  N(u);  h  =  L  h 

L  ^  J  T  T 

= y  w^(t,  s)  h  (s)  ds  +  r  r  r  ^2^^* 


h(s)  ds 


So  that,  taking  A  as  the  identity  matrix,  and  m^,  and  m^  as  Lebesgue  measure, 

we  have  _ 

»Tr  r»T  -ir/^T 


=  J  W2(t,  s,  a)  u(a)daj  Wj(t,  a>u(CT)  da 

T  T 

I  I  ^2^*'  ^^^*2  ] 

r\  -J' 

1 


(29) 


o  o 

-  f  +  WgU,  s,  o)  u(a)  da  |  g(t)  dt 


+  k  u  (s) 

Setting  this  equal  to  zero,  we  have  a  nonlinear  integral  equation  to  solve  for 
obtaining  a  local  extremum.  On  the  other  hand,  since  the  explicit  functional 
representation  x(t)  in  terms  of  u(t)  is  known,  the  steepest  descent  procedure 
can  be  applied  directly. 
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When  u(t)  is  of  dimensionality  higher  than  one,  the  representation  of  N{u) 
by  an  approximating  polynomial  is  somewhat  more  complex.  A  polynomial  of 
degree  m  being  written  as  the  sum  of  honiogeneous  polynomial  of  lower  degree 
will  be: 


N(u)  =  S  f  •••f  (t,  u(8j).  u^Sg). .  .u(Sj^)  )  ds^.  .  .dSj^  (30) 

k=o  '^o  ^  o 


where  for  each  t 


is  a  vector,  each  component  of  which  is  a  continuous  K-linear  form  in  the 
vector  x^,X2,  .  .  The  Frechet  derivative  can  again  be  calculated  in  a 

similar  manner. 

If  the  system  is  described  in  terms  of  the  nonlinear  differential  Equation 
(21),  it  is  possible  to  obtain  6[N(u);  h]  as  a  solution  to  a  linear  time  varying 
equation.  For,  in 

X  =  F(x,  u  +  X  h)  (31) 


for  fixed  u  and  h,  we  have  a  function  of  X  for  each  t,  and  at  X  =  0 


(32) 


where  VjjF(x,u)  and  VyF(x,  u)  are  the  gradient  matrices.  Now  (32)  is  a  linear 
dx 

equation  for  ^  ,  and  taking  initial  condition  for  (21)  to  be  zero,  so  that 


d  x(t) 
dX 

we  have 

dx 
dX 


=  0  for  t  =  0,  X  =  0 


=  J  <t>  (t)  $  (s)  V^F  (  x(s),  u(s)  ^  h  (s)  ds 


(33) 


where  $(t)  is  the  fundamental  matrix  solution  of 


Y  =  V^F[x,  u]  Y 
From  (33),  it  follows  that 
L  h  =  6t  N(u)  ;  h] 
corresponds  to  the  operation 
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x(s),  u(s)  J  h  (s)  ds 

To  use  this,  x(s)  must  be  known,  so  that  if  one  solution  is  known,  we  can 
perturb  it  to  obtain  a  better  approximation  to  the  minimization  problem  using 
Equation  (27).  Some  simplification  in  these  considerations  occurs,  if,  as  is 
usually  the  case, 

F(x,  u)  =  f^(x)  +  f^  (u) 

so  that 

V  F(x,u)  =  V  f,(x) 

X  X  1 

V^F(x,u)  = 


(s) 


V  F 
u 


VII.  eOMPUTER  STUDIES 


In  this  section  we  shall  present  the  results  of  some  computer  studies  of 
the  steepest  descent  methods  espoused  in  the  paper.  These  applications  are 
not  meant  to  be  exhaustive  or  even  representative  but  rather  to  illustrate 
some  of  the  specific  considerations  that  arise  in  such  application. 

Example  1.  Let  us  now  consider  a  second  order  linear  system  which  is 
described  by  the  differential  equation 


^  A 

d  X  dx 
dt2 


u(t) 


Assume  that  the  desired  output  over  the  control  interval  [0,  1]  is  a  step 
function,  i.e.. 


X  ,(t)  =  1  0  S  t  S  1 

a 


Let  the  initial  conditions  be 


x(0)  =  1  x'  (0)  »  -1 
Then 

X  (t)  =  e  ^ 
o 


Sind 
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g(t)  =  1  -e 


-t 


The  problem  then  is  to  have  the  position  output  of  the  system  following  the 
step  function  subject  to  the  energy  constraint  that 


2 


This  is  a  one -dimensional  problem. 

The  weighting  function  of  the  system  is 
W(t)  =  1  -e‘^ 


Then 

-d-T  )  -d-T  )  -(2-T,-TJ 

G(T  T  )  =-Max(T  T  )+e  ^+e  ^  _  1^  ^  ^  1 


+e 


1  '2' 


and 


L*g  =  0.  3678  -  T  -0.  5  e’^  +  0.  3001  e"*^ 


Here  Max  (T  ,  T  )  denotes  the  maximum  of  T  and  T^. 

1  <5  12 


In  carrying  out  the  computation,  we  take  6  =  0.  2.  The  initial  guess  for 
the  solution  is  arbitrarily  chosen  to  be  UQ(t)  =  1.  Some  of  the  computational 
results  are  listed  in  Table  I.  At  each  iteration  step,  we  have  to  adjust  k 
such  that  ” 

2 

2.8  S  II  u  II  S  3 

n  '' 


The  increment  of  adjustment  for  k^  in  this  problem  is  chosen  to  be  0.  005.  It 
is  interesting  to  see  that,  after  the  first  iteration,  the  number  k  is  already 
very  close  to  its  optimal  value.  At  the  third  iteration,  kj^  has  reached  its 
optimal  value. 


The  closeness  of  the  approximate  solution  for  u(t)  to  its  optimal  one  can 
be  measured  by  the  ratio 


ER  = 


(R  +  k  )  u  -L 
n  n 

* 

II  L  g  IL 


Thus  at  the  4th  iteration  with  ER  =  0.  0044,  we  can  stop  the  process  .  The  sys¬ 
tem  perforriiance  is  plotted  in  Figure  1  and  the  sequence  of  control  inputs  is 
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TABLE  I 

COMPUTATIONAL  RESULTS  B^OR  EXAMPLE  1 


2 


No.  of 
Iteration 

Steps 

n 

k 

n 

Ifvll' 

2 

t(R+k._)a„-I.*g|l2 

Q(u^) 

2 

Q(u  )+k  ||u  II 
n  n  ^2 

II  L  u  -  g|| 

n  j 

2 

hWl 

1 

0.006  5 

2.949360 

0  .  5150  50 

-0. 155592 

-0.  136421 

0.074357 

2 

0.0060 

2.  859088 

0.069710 

-0. 159816 

-0.  142657 

0.049229 

3 

0.  0055 

2.996001 

0.024473 

-0. 160658 

-0. 144180 

0.044222 

4 

0.0055 

2.969426 

0. 004438 

-0.  160531 

-0.  144200 

0. 044976 

5 

.  0.0055 

2.999596 

0.005363 

-0. 160701 

-0.  144203 

0.043963 

6 

0.  0055 

2.992307 

0.000992 

-0.160662 

-0.  144205 

0.044196 

7 

0.0055 

2.  999195 

0.00 1222 

-0. 160700 

-0.  144205 

0.043969 

8 

0.0055 

2.997514 

0.000226 

-0.160691 

-0.  144205 

0. 044024 

9 

0.0055 

2.999088 

0.000279 

-0. 160700 

-0.  144205 

0.043973 

10 

0.0055 

2.998706 

0.000052 

-0.  160698 

-0.  144205 

0.043985 

32 


t 

FIGURE  1.  performance  OF  THE  SYSTEM  IN  EXAMPLE  1 


33 
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plotted  in  Figure  2.  It  should  be  noticed  that  the  oscillation  of  after 

has  reached  its  steady  value  of  0.  0055  is  due  to  the  variation  of  ||  u(kp)  ||2  . 

However,  Q(Ujj)  +  k^^  11  indeed  a  non-increasing  sequence  for  the  same 

values  of  k  . 

n 

Example  2  ,  Let  us  consider  the  same  system  as  given  in  Example  1 . 
However,  we  shall  now  try  to  control  both  the  position  and  velocity  outputs  of 
the  system  to  follow  the  ideal  step  function  by  using  a  single  control  input.  In 
this  case,  we  have  a  2  x  1  system.  Thus 


g(t)  =  X  .(t)  -  x  (t) 

do 


Then 


-d-T  )  -U-T  )  -(2-T  -  T  ) 

G(Tj,  Tg)  =  Max  (T^.  T^)  +  e  ^  +e  ^  -e 

L*g  =  0.3678  -  T  -  0,2324  e^ 


The  constraint  for  the  input  is  taken  to  be 

II  u  If  g  5 

2 

In  carrying  out  the  computation,  we  again  choose  6  =  0.  2.  Hence  when 

II  u  II  exceed  5,  we  shall  adjust  k^  such  that 
2  " 
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TABLE  II 

COMPUTATIONAL  RESULTS  FOR  EXAMPLE  2 


2 


No.  of 
Iteration 
Steps 
n 

k 

n 

2 

II  u  II 

^"2, 

||(R+kn)u^-L*g||2 

Q(u„) 

2 

Q(u  )+k  II  u  II 

^  n  n" 

l|Lu^-g|| 

II  *  II 

|[L  g|[2 

^  llgll' 

1 

0 

2.515420 

0.396180 

-0.  4377  27 

-0,  437727 

0.396180 

2 

0 

1.970645 

0.  241534 

-0.  487548 

-0.487548 

0. 241534 

3 

0 

3. 240699 

0.196627 

-0. 513064 

-0. 513064 

0. 145496 

4 

0 

3. 295242 

0.  126567 

-0. 527287 

-0. 527287 

0. 121808 

5 

0 

. 4. 184334 

0.112038 

-0. 535891 

-0.535891 

0.  107479 

6 

0 

4.  307971 

0.  079227 

-0. 541763 

-0.  541763 

0.097699 

7 

0 

4.  975983 

0.078128 

-0, 546090 

-0. 546090 

0.090491 

8 

0.001 

4.  978586 

0.  059593 

-0. 548912 

-0. 544033 

0.085792 

9 

0.008 

4.  866785 

0.030549 

-0. 549684 

-0.  5107  50 

0.084506 

10 

0.005 

4.961765 

0.  029163 

-0. 551068 

-0.  526259 

0.082201 

11 

0.009 

4.953561 

0.018552 

-0. 551326 

-0.  516744 

0.081772 

12 

0.008 

4. 908812 

0.014031 

-0. 551118 

-0.510848 

0.082118 

13 

0.008  j 

4. 984433 

0.012147 

-0. 551844 

-0.511969 

0.080909 

14 

0.008 

4.  955171 

0.  011334 

-0.  551709 

-0.  512068 

0.081134 

0.009 

4.927553 

0.008125 

-0.  551580 

-0. 507232 

0.081348 

16 

0.008 

4. 942081  1 

0.009304 

-0.  55177  5 

-0. 512238 

0.081024 

17 

0.009 

4.  947917 

0.005447 

-0.  551861 

-0.  507330 

0. 080880 

18 

0.009 

4. 919139 

0.006306 

-0.  551625 

-0.507353 

0.081273 

19 

0.009 

4. 933637 

0.003887 

^  -0. 551774 

-0.  507371 

0.  081026 

20 

*  0.009 

4.914428 

0.004794 

-0.  551614 

-0.507384 

0.081291 

21 

Q.009 

4. 930254 

0.  003020 

-0. 551767 

-0. 507395 

0.081036 

22 

0.009 

4.916224 

0.  003880  . 

.  -0.551650 

:  -0.  507404 

0.081232 

23 

0.009 

4.  930914 

0.002415 

-0. 551789 

-0.507411 

0.081000 

24 

0.  009 

4.  920170 

0.003067 

-0.  551698 

-0.  507416 

0.081152 

25 

0.  0G9 

4.  933092 

0.001959 

-0.  551819 

-0.  507422 

0.080951 
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4.  8  S  II  u  |  £  5 

2 

The  increment  of  adjustment  for  this  problem  is  taken  to  be  0.  001.  The 
initial  guess  is  still  chosen  to  be  vi^(t)  =  1.  The  computational  results  are 
listed  in  Table  II.  When  are  slightly  different,  sometimes  show 

oscillations  as  in  Steps  No.  11  and  No.  12,  or  in  Ste^s  No.  14  and  No.  15. 

This  is  due  to  the  fact  that  we  are  not  adjusting  ||  uH^to  be  exactly  equal  to  5 
and  it  is  not  practical  and  necessary  to  do  so.  It  is  seen  that  larger  norm 
always  corresponds  to  smaller  Q.  Thus  the  finer  the  adjustment  for  k^  and 
the  smaller  the  value  for  6,  the  better  will  be  the  monotonocity  for  Q. 

Figure  3  shows  the  system  performance  and  Figure  4  shows  the  sequence 
of  controls.  It  is  seen  that  the  iteration  process  can  be  Stopped  at  the  15th 
step  with  ER  =  G.  008.  In  the  practical  application  of  this  method  when  on-line 
computation  is  required,  a  proper  threshold  level  for  ER  will  be  chosen.  The 
choice  of  this  threshold  should  be  based  on  the  desired  accuracy  of  the  solution 
and  the  computing  time  whicph  we  can  afford. 
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FIGURE  3.-  PERFORMANCE  OF  THE  SYSTEM  IN  EXAMPLE  2 
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SYSTEM  IN  EXAMPLE  2 


REFERENCES 


1.  Neustadt,  L.  W.,  "S5mthe sizing  Time  Optimal  Control  Systems/*  J.  of 
Math.  Anal,  and  Applic.,  Vol.  1,  No.  4  (1960),  pp.  484-493. 

2.  Ho,  Y.  C.,  “A  Successive  Approximation  Technique  for  Optimal  Control 
Systems  Subject  to  Input  Saturation,”  Trans.  ASME,  Series  D,  J,  of 
Basic  Engineering.  Vol.  84,  No.  1  (1962),  pp.  33-40. 

3.  Balakrishnan,  A.  V.,  “On  a  General  Theory  of  Nonlinear  Estimation 
Problems  in  Control  Systems,”  invited  lecture.  Symposium  on  Mathe¬ 
matical  Problems  in  Control  Systems.  Washington,  D.  C.,  November, 
1961. 

4.  Balakrishnan,  A.  V.,  "An  Operator-Theoretic  Formulation  of  a  Class  of 
Control  Problems  and  a  Steepest  Descent  Method  of  Solution,”  to  be 
published  in  J.  SIAM. 

5.  Kantorovich,  L.  V.,  “Functional  Analysis  and  Applied  Mathematics,” 
Usp.  Mat.  Nauk,  Vol.  3,  pp.  89-185. 

6.  Hille,  E.  and  R.  S.  Philips,  “Functional  Analysis  and  Semigroups,” 
American  Mathematical  Society  Colloquium  Publications.  Vol.  31,  1957. 


ASD-TDR-63-119 


The  Problem  of  Optimal  Mode  Switching 

By 

R.  A.  Nesbit 


University  of  California,  Los  Angeles 
Aerospace  Corporation, 

El  Segundo,  California 


41 


THE  PROBLEM  OF  OPTIMAL  MODE  SWITCHING 
R.A.  Nesbit 

University  of  California,  Los  Angeles 
Aerospace  Corporation,  El  Segundo,  California 


ABSTRACT 

Control  systems  may  be  designed  to  operate  in  different  modes.  In 
some  cases  these  modes  may  be  switched  to  obtain  various  system  trajec* 
tories.  The  optimization  of  the  trajectories  can  be  considered  as  a  deter¬ 
mination  of  the  switching  times  and  other  mode  parameters.  The  optimiza¬ 
tion  problem  is  to  determine  these  times  and  parameters.  First  order 
computations  can  yield  a  descent  method  but  convergence  is  not  assured. 
Theorems  which  deal  with  the  existence  of  solutions  to  optimal  control  prob¬ 
lems  which  have  been  recently  proved  indicate  that  it  is  better  to  relax  the 
dynamic  restrictions  during  the  optimization  than  to  work  only  with  the 
admissible  class  of  switched  controls.  The  resulting  optimal  trajectory  is 
then  approximated  by  one  of  the  trajectories  from  the  admissible  class. 

INTRODUCTION 

Some  vehicles  have  control  systenns  which  operate  in  various  modes, 
such  as  an  angle  of  attack  hold  mode,  a  bank  angle  hold,  a  flight  path  angle 
hold,  and  a  pressure  altitude  hold.  For  the  control  of  a  re-entry  vehicle 
one  might  include  other  hold  modes  such  as  acceleration,,  attitude,  thrust, 
or  temperature.  One  approach  to  trajectory  optimization  is  to  select  the 
appropriate  combination  of  modes  at  each  time.  For  some  optimization 
criteria,  however,  the  optimum  switching  will  "chatter”  or  change  rapidly 
from  one  position  to  another.  This  type  of  behavior  has  already  been  noted 
in  connection  with  relay  controllers.  A  necessary  condition  for  the  existence 
of  chattering  in  optimal  controllers  can  be  based  on  the  application  of  the 
maximum  principle.  This  condition  turns  out  to  give  information  about  the 
optimal  trajectory  in  trivial  exanrlples,  although  it  is  questionable  whether 
or  not  this  result  is  of  practical  significance.  Warga,  Roxin,  Filippov,  and 
Chang  (References  1  through  4),  deal  with  the  problem  of  existence  of  solu¬ 
tion  for  the  general  control  problem.  This  general  theory  seems  to  indicate 
that  optiniization  of  the  mode  switching  problem  should  be  accomplished  by 
generalizing  the  problem.  The  method  consists  of  first  relaxing  the  restric¬ 
tions  on  the  switching,  finding  the  optimum,  and  then  approximating  this 
optimum  by  the  restricted  switching  trajectories. 

A  sub-optimal  trajectory  can  be  constructed  using  a  fixed  switching 
sequence  and  adjusting  the  switching  times  by  a  steepest  descent  procedure. 

DISCUSSION 

A  class  of  mode  switching  systems  will  be  considered.  These  systems 
will  be  represented  by  the  vector  equations 
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dx 

ar  = 


a  ^  ®  ®o 

X  (tj)  €  Bj 

For  each  admissible  u  (t)  the  solution  of  the  differential  equation  is 
assumed  to  be  well  defined".  The  initial  condition  on  the  n-dimensional  state 
vector  is  in  some  region  Bq  and  the  terminal  condition  is  required  to  be  in 
some  region  The  r -dimensional  control  vector  u  is  chosen  from  a  set 

of  functions 


^  (x  -  £v)  ^  2.  .  .  m 

(2) 

r,  constant  vectors 
— k 

The  value  of  u  is  chosen  from  the  set  of  possibilities.  This  method  of 
formulation  is  ma3e  in  an  attempt  to  incorporate  the  design  restrictions 
into  the  optimization  problem.  Instead  of  formulating  a  completely  "open 
loop"  set  of  dynamics,  a  set  of  closed  loop  systems  is  considered.  The 
formulation  is  motivated  by  the  current  practice  of  designing  control  systems. 
For  example,  the  attitude  control  system  of  some  satellites  have  different 
sensor  and  feedback  configurations  for  acquisition,  earth  pointing,  and 
eclipse  operation.  The  application  of  these  systems  requires  some  sort  of 
switching  logic,  but  in  many  cases  the  logic  is  very  simple.  Since  the  sys¬ 
tem  designer  is  generally  willing  to  sacrifice  some  performance  for  a 
simple  mechanization,  it  is  of  interest  to  explore  the  possibilities  of  mode 
switching  logic. 

A  particular  application  which  motivates  this  study  is  the  trajectory 
control  of  a  vehicle,  and  while  this  problem  has  been  explored  in  some 
detail  to  find  optimum  trajectories  and  means  of  following  them,  it  seems 
reasonable  to  consider  "sub-optimal"  trajectories  if  they  are  easier  to 
inaplement. 

One  requirement  of  the  trajectory  problem  is  to  transfer  the  system 
from  an  initial  state  in  some  region  Bq  at  tQ  to  a  terminal  state  in  another 
region  B2  at  some  reasonable  time  t^  ^  t^*  The  fundamental  question  about 
whether  or  not  the  vehicle  is  capable  of  making  the  trip  is  of  interest  to  those 
concerned  with  the  journey.  To  satisfy  doubts  on  this  subject,  it  is  worth¬ 
while  to  find  at  least  one  control  sequence  which  makes  the  trip  possible. 

If  the  journey  is  so  difficult  that  only  one  control  scheme  will  work, 
then  the  question  is  how  to  find  and  mechanize  it.  But  if  there  are  many 
possibilities,  then  some  choice  of  route  must  be  made  even  if  it  is  made  by 
chance.  In  some  "games"  random  selection  is  the  "optimal  strategy^*;  in 
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other  cases  there  is  some  "cost"  which  is  to  be  minimized.  Again,  existence 
and  uniqueness  are  of  interest.  When  there  is  a  control  which  completes  the 
journey  at  some  cost,  then  the  existence  of  a  minimizing  trajectory  is  of  less 
importance  since  the  cost  can  be  made  reasonably  close  to  the  minimum. 

The  lack  of  uniqueness  of  the  optimizing  trajectories  merely  requires  an 
additional  decision.  Some  writers  indicate  that  the  problem  "is  not  properly 
formulated"  as  long  as  any  ambiguity  remains,  but  even  without  admitting  this 
one  can  conceive  of  a  hierarchy  of  "play-offs"  leading  to  the  final  choice  of 
controls.  For  this  discussion  the  first  component  of  the  state  vector  xl  (tj) 
at  the  terminal  time  t|  will  be  taken  as  the  cost  function  to  be  minimized. 

THE  SWITCHING  CONCEPT 

The  control  problem  is  to  select  one  of  the  possible  switch  positions. 

A  block  diagram  of  the  system  described  above  is  shov/n  in  Figure  1. 


Figure  1.  Block  Diagram  of  Mode  Switching  System. 
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The  switching  logic  is  required  to  give  a  trajectory  which  satisfies 
terminal  conditions.  For  linear  systems  this  requirement  may  be  imple¬ 
mented  by  a  final  value  controller  (Reference  5).  The  significant  computa¬ 
tion  carried  out  in  the  final  value  control  problem  is  the  conditional  response 
prediction.  This  computation  utilizes  the  known  dynamics  of  the  system  to 
extend  the  trajectory  to  some  future  time  under  the  assumption  of  a  particu¬ 
lar  control  sequence. 

This  idea  is  applicable  to  nonlinear  as  well  as  linear  systems.  Let 
^  (t,  xq,  tQ)  designate  the  solution  of  the  equations  at  time  t  with  the 
imtiai~itate  xq  at  time  tg  and  using  the  mode  of  control.  This  solution 
is  a  conditional  response  predictor  if  the  present  state  is  used  as  initial 
condition  and  if  the  terminal  time  is  used.  That  is, 

x(tj)  =  x(t),  tj 

if 

u  =  ^  (x)  for  t  ^  T  ^  tj 

The  system  may  start  in  mode  j  and  switch  to  mode  k  at  time  t^  in 
this  case 

*(^2^  =  ^0’.  *0^’ 

The  effect  of  small  changes  in  the  switching  time  tj  are  of  interest  and 
can  be  estimated  on  the  basis  of  first  order  computations. 

Computation  of  the  First  Order  Influence  of  Switching  Time 
and  Initial  StAte  ^ 


The  computation  of  the  first  order  effects  of  changes  in  ti ,  xq,  and  tQ 
on  0  (ti  XQ,  tQ)  can  be  made  for  the  general  mode  and  the  mode  subscript 
will  be  dFopped.  The  first  order  effects  describe  deviations  from  some 
” nominal"  trajectory.  The  deviations  from  the  nominal  will  be  designated 
by  the  vector  z,  and  the  nominal  by  the  vector  y(t)  with  y(tQ)  =  xq.  Then 
z  =  X  -  y  and  Hie  dynamics  of  the  deviation  are"”  “ 

z  =  f  (y  +  2)  -f(Y)  (3) 

Assuming  that  the  first  partial  derivatives  of  f  are  continuous,  define 
the  matrix  A  with  elenients 


a]  (t) 


(4) 


As  usual,  the  fundamental  solution  of  the  adjoint  equation  will  be  used. 
Define  the  matrix  W{t )  such  that 
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(5) 


dW 

■ar  = 


-  WA 


W(tj)  =  I 


By  combining  Equations  (3)  and  (5)  one  may  obtain 
^  ( Wz)  =  W  [f  (y  +  z)  -_f  (y)  -  Az] 

and  thus 

^(Wz)  =  0|z|^ 

where  0  |z^||^  indicates  the  terms  of  second  and  higher  order.  Integrating, 
one  obtains 

f‘l  2. 

W(tj)z(tj)  =  W(tQ)ZQ+  0||z||  dT 

K 

or 

i(‘r  iO'  V  = 

Thus,  to  a  first  order  of  approximation  in  |  z  |  ,  one  obtains 

z(tj)  =  W'*  (tj)  W  (tp)  Z  (tg)  (7) 

This  will  be  designated  by 

£(tj)  =  K(tj,  t^)  z(t^)  (8) 


A  first  order  approximation  to  the  effect  of  switching  time  on  the  final  state 
can  be  obtained  by  considering  the  equivalent  perturbation  Az  due  to  a  change 
in  switching  time  and  propagating  the  perturbation  to  the  final  time.  For 
switching  from  the  mode  to  the  mode  this  perturbation  is 


A*  (tj)  -  [l(x,  Uj)  -  £  (x,  ^)]  At. 

and  the  effect  on  the  final  state  is 

Az(t^)  =  K  (t^.  t.)  Az  (tj) 


(9) 


(10) 
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Defining 


Vi  =  K  (t^,  t.)  [f  (X,  u.)  .^(x,  ^)]  (11) 

one  may  write  the  final  perturbation  vector  as 


li  (t„)  K  (t„. 


nr  1 

‘q)  +  I-  li 

ir  I 


(12) 


A  linearized  mode  switching  controller  could  be  designed  by  setting  z(tj^)=  0 
and  solving  for  At^  using  a  psuedo  inverse  if  necessary.  The  effect  of  small 
changes  in  the  switching  times  Ati  can  be  used  to  "improve"  the  nominal  by 
the  usual  steepest  descent  method.  The  switching  influence  vectors  ^  may 
span  a  subspace  of  the  state  space  z  even  if  the  number  of  switching  times 
is  large. 

At  each  step  of  a  descent  procedure  the  values  of  Ati  must  be  chosen. 
This  can  be  done  by  making  At^  the  "solution"  to  the  equation 

V.  At,  +  V,  At,  +  .  .  .  +  V  At  r  Az  (t  ) 

— 1  1  ^  Z  — s  8  — '  n' 

using  a  psuedo  inverse  if  necessary. 

This  procedure  will  not  converge  in  many  cases,  and  the  real  problem 
is  to  determine  the  conditions  under  which  it  will  converge.  A  fairly  trivial 
example  will  now  be  discussed  to  illustrate  the  descent  procedure. 

EXAMPLE 


The  computations  of  the  descent  procedure  will  be  illustrated  in  the 
usual  first  order  system  with  mean  square  error.  The  system  is  shown  in 
Figure  2  and  is  described  by 


=  (X  -  1)‘ 


.2 

X 


{ 


'(1-q)  (x‘*+2) 
-a  2x^ 


a  =  0  or  1 


x^O) 


x^(0)  =  -  0.5 
x^(2)  =1.5 
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Figure  2.  System  for  Example. 


As  will  be  shown  later,  this  system  will  be  required  to  chatter  in  order 
to  approach  the  non -attainable  minimum  of  xl(2).  During  this  chattering 
the  process  will  "slide”  along  the  trajectory  x2  -  1.  For  the  present,  a 
sub-optimal  trajectory  with  only  two  switches  will  be  considered..  The 
switch  parameter  a  will  be  taken  as 

0  for  0  <  t  ^  t  j 

1  for  t  j  <  t  ^  t^ 

0  for  t^  <  t  ^  2 

( 

and  the  nominal  trajectory  will  switch  at  t^  =  0.5  t^  =  1*5. 


The  adjoint  equations  are 


"^11 


^21  ^22 


1*'ZI  '*'2dr  -2Q  +  (l-a) 


and  since  does  not  depend  upon  these  equations  may  be  solved  along 
the  nominal 


W3  . 


I  (‘2)  +  ^  l«*p  2  •  *2^  ■  il 

+  6  [1  -  exp  (2  -  t^)] 


exp  (2  -  t2) 


[0  exp  -  2  (t^  -  tj) 

The  velocity  difference  given  by 


Along  the  nominal  mentioned  above 


'-1.62\ 

.0.762/ 


r-0.  766' 


■  3.  61 


Thus  the  switching  time  increments  are  related  to  the  changes  in  the 
terminal  state  through  the  equation 


-1.62 

0.  766 

Ax^2) 

0.  762 

-3.  61 

1 

> 

1  ^ 

— 

Ax^(2) 

to  a  first  order  approximation  in  Atj^  and  Axi(t),  The  nominal  trajectory 
does  not  quite  satisfy  the  terminal  conditions,  and  the  increment  in  terminal 
state  is  taken  as 


Ax'' 

-0.  1 

Ax^ 

II 

^0  .  07_ 

This  choice  gives  switching  increments  of  At^  =  0,  0556  At2  =  -  0.  0133. 

The  modification  of  the  switching  times  does  improve  the  trajectory. 
Repeated  application  of  the  procedure  gives  the  sequence  of  trajectories 
tabulated  in  Figure  3. 
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♦o 

t, 

»^<t,  ) 

♦3 

«2(f3) 

NOM. 

0 

^.5 

0.5 

0.437 

1.500  0.0640 

2 

1.403 

TRIAL  1 

0 

"0.5 

0.537 

0.566 

1.481  0.0857 

2 

1.504 

TRIAL  2 

0 

-0.5 

0.585 

0.692 

1.494  0.1123 

2 

1.503 

t  (TIME) 


Figure  3.  Sketch  of  Trajectories. 


The  steepest  descent  procedure  outlined  above  can  be  used  in  more 
complicated  examples,  and  the  principle  saying  of  computation  and  storage 
is  due  to  the  fact  that  the  influence  matrices  only  need  to  be  stored  at  the 
switching  instants.  If  two  switching  times  t  and  z  approach  each  other 
nearer  than  some  specified  e,  that  is  if  jtj  ^  t2|  <  €  the  intermediate  mode 
may  be  omitted  and  the  number  of  switches  reduced.  This  procedure  leads 
to  a  sub-optimal  policy  in  the  mathematical  sense,  but  there  are  often 
practical  reasons  for  avoiding  a  large  number  of  switches.  The  available 
knowledge  about  the  desired  trajectory  and  mode  properties  can  be  used  to 
select  a  reasonable  nominal  sequence. 
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A  Necessary  Condition  for  Optimal  Switching 

The  system  described  by  Equations  (1)  and  (2)  has  the  following  adjoint 
to  the  first  variation. 


gf’^  ^  ™  gf^  gu^ 

i  ^  .  a  r 

ax  as  1  gu  ax 

The  Hamiltonian  is 


For  a  minimizing  trajectory  the  maximum  principle  (Reference  6) 
requires  that  the  gj^  be  chosen  to  maximize  H  for  suitably  chosen  boundary 
conditions  on  the  multipliers  p^.  As  an  alternative  to  steepest  descent  com¬ 
putation  of  a  fixed  number  of  switching  times,  a  steepest  descent  procedure 
for  determining  the  initial  conditions  on  the  multipliers  may  be  attempted. 
The  relative  merits  of  these  two  procedures  are  of  interest.  One  problem 
which  may  arise  in  the  optimal  switching  case  is  the  problem  of  chattering. 
The  necessary  condition  outlined  above  gives  an  unambiguous  specification 
of  the  switch  position  when  one  of  the  quantities  p  •  ^  (x,  is  actually 
gjr eater  than  any  of  the  others.  However,  when  mere  are  two  or  more  of 
these  quantities  equal  to  the  maximum,  then  the  choice  is  not  clear,  This 
equality  occurs  at  each  switching  instant,  and  if  the  equality  only  holds  for 
an  interval  of  measure  zero,  there  is  no  ambiguity  of  the  resulting  trajec¬ 
tory. 


Pi  = 


-ti 

j=  l  k=l 


If  the  multiple  maximum  persists,  then  the  switching  choice  is  not 
defined  by  the  maximum  principle,  although  it  may  possibly  be  defined  by 
\^the  persistence  of  the  multiple  maximum.  The  optimum  switching  choice 
may  not  exist  in  piecewise  continuous  form,  and  the  optimal  control  choice 
"chatters'*  between  the  two  or  more  possibilities.  In  this  case  the  sub- 
optinnal  choice  of  a  fixed  number  of  switches  may  be  more  desirable  com¬ 
putationally. 

A  more  rigorous  discussion  of  chattering  may  be  based  on  the  work  of 
Warga,  Roxin,  Filippov,  and  Chang  (References  1  through  4).  The  switching 
problem  is  first  associated  with  a  relaxed  problenn  by  considering  values  of 
gj^:  0  <  ^  I  =  1  instead  of  only  =  0  or  1,  =  1.  This  relaxation 

allows  the  velocity  vector  to  take  on  all  the  values  in  the  simplex  with 
corners  at  f  (^,  instead  of  only  the  value  of  the  corners.  For  the 
relaxed  prc^letn  a  solution  exists  as  shown  in  the  above  papers.  Warga 
also  shows  that  this  optimal  solution  can  be  approximated  arbitrarily  closely 
by  using  only  the  original  switching  control.  The  application  of  this  result 
to  the  problem  of  optimum  mode  switching  seems  to  indicate  that  the  relaxed 
problem  should  be  solved  for  the  optimal  trajectory  and  then  approximated 
by  the  switched  trajectory.  However,  if  one  is  going  to  attempt  to  follow  a 
nominal,  then  one  of  the  other  perturbation  techniques  (Reference  7)  may  be 
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more  feasible  for  closed  loop  operation.  It  appears  that  switching  modes 
does  not  simplify  the  optimum  trajectory  computation  unless  a  sub-optimal 
trajectory  with  a  fixed  number  of  switches  is  accepted. 

'  EXAMPLE 

The  dynamics  of  the  previous  example  are  used,  but  an  attempt  is  made 
to  find  the  "optimal"  switching.  The  multipliers  pi  satisfy  the  equations 


h  =  -  2  -  l)  Pj  -  P2  [(1  -  a^)  -  2a^ 

and  the  Hamiltonian  is 

H  =  Pi  (x^  -  1)^  +  P2  |(1  -  a^)  (x^  +  2)  -  2x^j 

This  quantity  is  a  constant  and  switching  can  only  occur  when  x^  =  -  2/3  or 
PZ  =  W  P2  ==  P2  *  ®  only  if  x^  =  1  and  thus  a  necessary  condition  for 
chattering  is  =  1 . 

CONCLUSION 

A  steepest  descent  modification  of  the  switching  times  in  a  mode  switch¬ 
ing  control  system  can  be  based  on  first  variation  computations,  and  the 
computer  storage  requirement  less  than  that  required  for  functional  descent. 
By  introducing  control  variables  the  mode  switching  system  can  be  put  in  the 
format  usually  used  in  the  study  of  control  optimization.  The  application  of 
the  maximum  principle  yields  a  necessary  condition  for  optimum  switching 
and  a  necessary  condition  for  chattering.  If  the  switching  problem  is  relaxed 
to  assure  the  existence  of  an  optimal  solution  the  computational  problem  of 
finding  this  optimum  is  not  simplified  by  the  switching  approach  although 
mechanization  of  the  control  system  may  be  simplified.  The  use  of  steepest 
descent  procedure  for  determining  the  sub-optimal  control  using  a  fixed 
number  of  switches  is  only  based  upon  differential  arguments  and  some 
knowledge  of  the  global  properties  of  the  trajectory  is  required  to  assure 
the  usefulness  of  the  procedure. 
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ABSTRACT 

The  problem  of  specifying  the  minimal  fuel  thrust  program  for  guiding 
a  space  vehicle  in  its  midcourse  phase  is  formulated  as  an  optimization 
problem.  The  task  of  the  thrust  program  is  to  transfer  the  vehicle  from  a 
given  initial  error  state  (position  and  velocity)  to  a  terminal  error  state  of 
zero  with  minimum  fuel  subject  to  a  magnitude  constraint  on  the  thrust  vector. 

The  form  of  the  optimal  thrust  program  is  derived  and  shown  to  be  of  an 
"on-off”  nature  with  time -varying  direction  during  periods  of  thrusting. 

The  problem  of  synthesizing  the  optimal  thrust  program  is  reduced  to 
the  problem  of  maximizing  a  particular  function  whose  gradient  is  readily 
computed.  An  iterative  computational  procedure  for  synthesizing  the  optimal 
thrust  program  is  then  developed  utilizing  the  method  of  steepest  ascent. 

INTRODUCTION 

Let  us  consider  the  problem  of  guiding  a  space  vehicle  during  the  mid¬ 
course  phase  of  its  mission.  Basically,  the  problem  is  that  of  correcting  the 
space  vehicle*8  ballistic  or  "free  fall"  trajectory.  In  general,  this  involves 
thrusting  at  low  levels  in  contrast  to  the  large  amounts  of  thrust  required  for 
launch  or  deboost  operations.  We  indicate  a  typical  mission  involving  a  mid- 
course  maneuver  in  Figure  1. 

There  are  two  primary  functions  which  a  midcourse  guidance  system 
must  perform.  The  first  is  one  of  sensing  and  processing  information  to 
determine  the  vehicle's  state,  such  as  position  and  velocity,  in  an  appropri¬ 
ate  coordinate  system.  This  is  the  navigation  function.  The  second  is  that 
of  utilizing  the  navigation  information  to  generate  thrust  control  signals  so 
that  mission  objectives  are  achieved.  This  is  the  thrust  programming  funcr 
tion. 


Numerous  problems  dealing  with  the  optimization  of  flight  paths  and 
guidance  systems  have  been  treated  in  the  literature  (References  1  through6). 
Here  we  shall  consider  the  specific  problem  of  minimal  fuel  thrust  program¬ 
ming  for  midcourse  flight  where  the  corrective  thrust  vector  is  amplitude 
constrained.  The  central  result  of  the  paper  will  be  the  development  of  an 
iterative  computational  technique  for  generating  the  thrust  program.  With 
respect  to  the  navigation  functions,  we  shall  assume  that  midcourse  guidance 
is  to  be  initiated  at  a  specified  instant  of  time  at  which  the  vehicle's  state  is 
known  to  a  desired  degree  of  accuracy. 
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Figure  1.  Typical  Space  Mission  Involving  Midcourse  Maneuver. 


THE  MINIMAL  FUEL  PROBLEM 

If  launch  or  deboost  operations,  that  is,  phases  of  a  space  mission 
involving  periods  of  relatively  high  level  thrusting,  could  be  executed  per¬ 
fectly,  a  given  space  vehicle  would  free  fall  to  the  desired  destination  along 
a  path  which  we  shall  term  the  nominal  trajectory.  However,  imperfect 
launch  or  deboost  operations  cause  errors  to  exist  along  the  ballistic  or 
free  fall  portion  of  the  niission.  We  use  the  term  errors  here  to  denote 
position  and  velocity  deviations  from  the  nominal  trajector^y.  We  assume 
these  errors  are  small  enough  to  permit  a  linearization  of  the  vehicle's 
equations  of  motion. 

To  within  the  accuracy  limitations  of  the  guidance  elements  utilized,  we 
shall  require  that  the  corrections  reduce  the  errors  to  zero  at  the  conclusion 
of  the  midcourse  phase.  We  shall  also  require  that  the  midcourse  phase  be 
executed  in  a  fixed  time  so  that  the  vehicle  will  arrive  at  the  same  terminal 
state  at  the  same  time  as  a  vehicle  following  the  nominal  trajectory.  By 
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imposing  these  two  requirements,  we  have  considerably  reduced  the  task  of 
the  terminal  guidance  system  in  executing  re-entry,  rendezvous,  or  docking 
type  maneuvers. 

In  order  to  maximize  the  useful  payload,  we  shall  choose  quantity  of  fuel 
as  the  system  parameter  to  be  minimized.  Moreover,  for  obvious  practical 
reasons,  we  shall  assume  the  corrective  thrust  is  amplitude  constrained. 

The  equations  of  motion  of  a  space  vehicle  subject  only  to  gravitational 
and  propulsive  forces  may  be  written  as 

y  =  P  (y,  t)  +  Q  (y)  T  (t)  (1) 

for  either  cartesian  or  spherical  coordinates.  In  Equation  (1),  y  is  a  six- 
dimensional  column  vector  whose  elements  are  the  position  and  velocity 
coordinates  of  the  vehicle,  P  is  a  six-dimensional  column  vector  represent¬ 
ing  the  gravitational  forces,  and  Q  (y)  is  a  6  X  3  matrix  which  relates  the 
three-dimensional  thrust  vector  T  to  y. 

Assuming  small  errors,  we  shall  now  linearize  Equation  (1).  We  let 
y  =  Y  +  6y  where  Y  is  the  nominal  trajectory  which  satisfies  the  free  fall 
equation 


Y  =  P  (Y.  t).  (2) 

The  initial  conditions  on  Equation  (Z)  are  those  which  would  result  frorn  an 
ideal  (perfect)  launch  or  deboost  operation. 

Since  y  satisfies  Equation  (1),  6y  satisfies  the  equation 

6y  =  6y  +  Q(y)  T  (t)  +  ,  (3) 

where  (3P/3y)  is  a  6  X  6  matrix  (evaluated  along  the  nominal  trajectory  Y), 
and  the  dots  on  the  right  denote  higher  order  terms  in  6y. 

Assuming  Q  (y)  is  approximated  closely  enough  by  Q  ( Y)  and  neglecting 
the  higher  order  terms.  Equation  (3)  becomes 


6y  =  6y  +  Q  (Y)  T  (t).  (4) 

Since  the  tirne  history  of  the  norninal  trajectory  is  known,  we  observe  that 
(dP/dy)  and  Q  (Y)  are  explicit  functions  of  time.  Hence,  defining 

x(t)  =  6y(t)  A(t)  =  gSj 

u  (t)  =  T  (t)  B  (t)  =  Q  (Y). 
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Equation  (4)  becomes 


X  (t)  =  A  (t)  X  (t)  +  B  (t)  u  (t) . 


(5) 


In  Equation  (5),  x  (t)  is  a  six -dimensional  column  vector,  the  vehicle  state 
vector;  u  (t)  is  a  three-dimensional  column  vector,  the  thrust  vector;  and 
A  (t)  and  B  (t)  are  6x6  and  6X3  matrices,  respectively. 

We  shall  let  the  time  interval  over  which  midcourse  guidance  is  per¬ 
formed  be  the  interval  0  ^  t  ^  T,  where  T  is  a  known  constant.  We  shall 
denote  the  initial  errors  or  deviations  from  the  nominal  trajectory  by  x  (0). 
The  task  of  the  midcourse  guidance  system  is  then  to  null  these  errors  so 
that  x  (t)  =  0. 

For  low-level  thrust  propulsion  systems  operating  in  free  space,  it  is 
reasonable  to  assume  the  specific  impulse  Igp,  given  by 


|T  (t)| 

'  ''  » 

S  zh 
»o 


(6) 


is  constant  (Reference  7).  In  Equation  (6),  [t  (t)|  is  the  magnitude  of  the 
thrust  vector,  go  is  the  acceleration  of  gravity  at  sea  level,  and  xh  is  the 
time  rate  of  change  of  the  fuel  mass.  Cross -multiplying  in  Equation  (6), 
and  integrating  the  result  between  the  limits  0  and  T,  we  obtain 


IT  (t)|  dt 


g  I  ,  m  dt 
»o  sp 


)T 

g  I  dm. 

®o  sp 


(7) 


8o  Jo  simply  the  total  weight  of  the  fuel  consumed  during  the  mid- 

course  {^se.  From  Equation  (7)  then,  we  see  that  the  time  integral  of  the 
thrust  vector  magnitude  is  proportional  to  the  fuel  consumed.  Therefore,  in 
order  to  minimize  the  fuel  consumed  during  the  midcourse  phase,  we  shall 
choose 


S 


|T  (t)|  dt. 


or  equivalently, 

"  :  s  =  f  i|u  (t)||  dt.  (8) 


as  the  design  parameter  dr  cost  to  be  minimized.  In  Equation  (8),  the 
symbol  |  |  denotes  the  Euclidean  norm  as  given  by 


II u  (t)||  =  (t)  +  (t)  +  (t) . 

This  is  simply  the  magnitude  of  the  thrust  vector. 

In  minimizing  Equation  (8),  we  shall  restrict  ourselves  to  thrust  vectors 
for  which:  (a)  ||u  (t)||  ^  1,  and  (b)  each  component  of  u  (t)  is  piecewise  con¬ 
tinuous.  Thrust  vectors  which  satisfy  these  two  restrictions  for  0  ^  t  ^  T 
are  termed  admissible. 

In  practice,  condition  (a)  is  equivalent  to  requiring  |T  (t)|  ^  Tq  where 
Tq  is  a  known  constant  equal  to  the  maximum  thrust  available  from  the  pro¬ 
pulsion  system.  For  analytic  simplicity,  we  have  normalized  this  constraint. 
Condition  (b)  is  one  of  requiring  that  the  thrust  vector  be  physically  realiz¬ 
able  . 


We  now  summarize  the  problem  formulation;  We  are  given  the  linear¬ 
ized  system  of  Equation  (5)  of  a  space  vehicle  in  midcourse  flight.  We  wish 
to  determine  an  admissible  thrust  program  u  (t)  which  will  transfer  the 
vehicle  from  a  known  initial  state  x  (0)  to  the  terminal  state  x  (t)  =  0  in  a 
fixed  time  t  such  that  the  cost  (Equation  (8))  is  minimized.  Such  an  admis¬ 
sible  thrust  program  will  be  called  an  optimal  thrust  program. 

DERIVATION  OF  THE  OPTIMAL  THRUST  PROGRAM 

For  a  given  initial  state  x  (0)  and  an  admissible  thrust  program  u  (t), 
the  state  of  the  system  (Equation  (5))  at  any  time  t,  0  ^  t  ^  T,  is  given  by 


x(t)  =  X(t) 


X  (0)  + 


.-1 


X  '  (s)  B  (s)  u  (s)  ds 


(9) 


where  X  (t)  is  the  6x6  matrix  solution  of  X  (t)  =  A  (t)  X  (t),  and  X  (0)  r  I  = 
identity  matrix. 

From  Equation  (9),  we  see  that  we  can  achieve  the  desired  terminal 
state  X  (t)  =  0  if  and  only  if 


-x(0) 


IT 

X’^s)  B  (b)  u  (s)  ds. 

Jo 


(10) 


Let  us  define  an  additidnal  state  variable  y  by 


y  (t) 


||u  (8)1  da. 


We  xu>te  frozn  Equation  (8)  that  S  -  y  (T). 
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Let  us  consider  the  set  0  defined  by  the  relation 


n  =•< 

1  X'Mt)B  (t)u(t)dt. 

I  11“  (t)||  dt 

:  u  (t)  admissible 

Jo 

Jo 

j 

We  see  that  0  is  a  set  in  seven-dimensional  Euclidean  space,  and  that  a 
point  (x,  y)  belongs  to  0  if  and  only  if  there  exists  an  admissible  thrust  pro¬ 
gram  u  (t)  which  transfers  the  initial  state  -x  to  the  origin  in  time  T  with  a 
cost  y.t 

It  is  shown  in  Reference  8  that  0  is  convex,  closed,  and  bounded,  and 
that  it  is  symmetric  about  the  y  (cost)  axis. 

For  a  given  x  (0),  the  minimum  cost  to  reach  the  origin  in  time  T  is  the 
least  y  (T)  for  which  (-x  (0),  y  (T))  €  We  shall  denote  this  minimum  cost 
by  yO  Clearly,  (-x  (0),  yO  (T))  is  a  boundary  point  of  Cl,  We  let 

3^  =  (-X  (0),  yO  (T)).  (Bold  faced  letters  will  be  used  throughout  to  indicate 
8  even -dimensional  vectors.) 

Since  Cl  is  convex,  there  exists  at  least  one  seven-dimensional  column 
vector  T]  such  that 


n' •  •  JK .  (11) 

for  all  !£  s  Cl  where  the  prime  denotes  the  transpose.  That  is,  we  may  con¬ 
struct  aThyperplane  of  support  to  n  at  with  t)  as  any  vector  normal  to 
this  hyperplane  at 

From  the  properties  of  Cl  it  can  be  shown  that  the  last  (seventh)  com¬ 
ponent  of  is  nonpositive.  With  little  loss  of  generality,  we  shall  assume 
it  is  negative.  Moreover,  since  the  length  of  is  immaterial,  we  shall  set 
its  last  component  equal  to  -1  for  all  the  work  which  follows. 

From  Equation  (11),  we  see  that  the  function  u  attains  its  maximum 
in  Cl  when  w  =  Xq.  For  any  w  c  Cl  given  by 

\pT 

w  =  I  X"^  (t)  B  (t)  u  (t)  dt, 
yo 

this  function  becomes 

^It  is  conceivable  that  for  sufficiently  large  initial  errors  x  (0),  it  would  be 
impossible  to  achieve  x  (T)  =  0  even  if  full  thrust  ||u  (t)(  =  I  were  utilized 
throughout  tte  interval  0  ^  t  ^  t.  In  pur  work  here,  we  shall  consider  only 
those  initial  errors  which  can  be  transferred  to  the  origin  in  time  t  using 
admissible  thrust  programs. 


||u  (t)||  dt 
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V  .  w  =  /  [t)*  •  (t)  B  (t)  u  (t)  -  |u  (t)|]  dt,  (12) 


where,  from  above,  r\  =  (t),  -1)  and  r\  =  (t)^,  .  .  t)^). 

We  wish  to  select  an  admissible  thrust  program  u  (t)  which  maximia^es 
n '  •  We  maximize  this  function  by  maximizing  the  integrand  of  Equation 
(l2)  for  all  t,  0  ^  t  ^  T.  Thus,  we  wish  to  maximize  the  function 


.-1 


f  (t)  =  n'  •  X'^  (t)  B  (t)  u  {t)  -  ||u(t)|| 


(13) 


with  respect  to  the  variable  u  subject  to  the  constraint  |  u||  ^  1. 

We  observe  that  r]'  *  (t)  B  (t)  is  a  three-dimensional  row  vector. 

If  q*  •  X“l(t)  B(t)  =  0,  we  maximize  f(t)  by  setting  u(t)  =  0.  If 
T^'  *  X*i  (t)  B  (t)  ^  0,  f(t)  will  be  maximized  only  if  u(t)  has  the  same 
direction  as  q'  •  X-1  (t)  B(t).  We  shall  assume  this  is  the  case  in  the  work 
which  follows.  Hence,  for  t)*  ^  X"l(t)B(t)^  0,  Equation  (13)  becomes 


f{t)  =  ||u(t)||  [in'  .  x-^t)B(t)||  -  i]. 


(14) 


If  •  X"^t)B(t)|j  >  1,  f(t)  is  maximized  by  making  ||u(t)|  =  1,  the 
maximum  allowable.  Since  u(t)  has  the  same  dir  ection  as  •  X"^  (t)  B  (t), 
we  obtain 


u(t) 


.1: 


1 


h’ 


X  Mt)  B(t)j 

:‘Nt)  B(t)jl 


If  Ht|*  •  X“^(t)  B  (t)||  <  1,  f  (t)  is  maximized  by  setting  ||u(t)|  =  0  which 
means  u  (t)  =  0. 

Finally,  if  Kr|*  •  X"1  (t)  B  (t)||  =  1,  we  have  f  (t)  *  0  for  any  admissible 
u(t).  For  purposes  of  our  work  here,  we  shall  assume  that  the  set  of  points 
in  the  interval  [O,  at  which  ||r|*  •  X*^  (t)  B  (t)||  =  1  is  of  measure  zero  for 
every  vector  T).  For  the  sake  of  completeness,  however,  we  set  u(t)  =  0 
whenever  \r\  •  X“l(t)B(t)|  =  I.  For  the  above  assumption  then,  we  have 
that  the  thrust  program  u  (t)  which  maxinriizes  r]'  •  o)  is  unique  and  defined 
almost  everywhere. 

Because  of  the  uniqueness,  we  can  replace  inequality  (11)  by  the  strict 
inequality 


n*  •  X  >  n  *  •  u),  0)  €  n  and  w  ^  x_ 

MW  MW  MW 


(15) 


In  summary,  the  thrust  program  which  maximizes  t)'  *  ta  (and  thereby 
minimizes  the  cost)  is  given  by  the  relation 
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u(t)  =  \ 


(fa  • 

h*.  3C-'WB(t)| 


"Whenever  |t|*  «  ft)  B  (t)|  >  1 


whenever  ||tj'  •  X"^(t)  B(t)||  ^  I, 


(16) 


for  0  s  t  T. 

SYNTHESIS  OF  THE  OPTIMAL-  THRUST  PROGRAM 

We  see  from  Equation  (16)  that  the  problem  of  generating  the  optimal 
thrust  program  is  one  of  computing  an  T)  for  a  given  initial  state  x(0). 

Let  us  consider  a  seven-dimensional  column  vector  (X,  -1)  where 
X  s  (Xj,  .  .  . ,  kff).  For  a  given  X,  let  us  define  the  functi<m 


v(t.  X)  = 


[\'  •  ^±*1.  whenever  |x'  •  X"^(t)B(t)(  >  1 

|X'  •  X’Mt)  B(t) 


0  whenever  |X'  •  X'^  <t)  B(t)||  S  1, 

for  0  s  t  s  T.  Let  us  introduce  the  vector  function 


(17) 


*,(X)  = 


X’  ^  (t)  B  (t)  V  (t,  X)  dt,  /  ||v  (t.  X)|  dt 


(18) 


Analogous  to  Equation  (15),  we  obtain 

X*  •  £(X)  >X*  •  C  e  fl  and  C  #z(X).  (19) 

Hence,  s(X)  is  a  boundary  point  of  0  and  a  hyper  plane  of  support  to  0  at 
^(X)  has  the  normal  vector  X  =  (X,  -1). 

Mofw  if  a  vector  q  is  known  for  a  given  initial  state  x(0),  the  corres¬ 
ponding  point  on  the  boundary  of  H  has  the  coordinates 

z(Tj)  =  (-x(O),  y°(T)).  (20) 

Let  us  consider  any  vector  X  for  which  ^(X)  ^  JI(ti)*  We  shall  denote 
the  hyperplane  of  support  to  0  at  ^(X)  by  B  (X).  If  ($1,  .  .  .  ,  Sg,  §7)  is 
any  point  in  the  seven -dimensional  Euclidean  space  which  also  lies  in  B(X)9 
we  obtain 


i'  •  ifr)  =  V  ■  1 


as  the  equation  for  this  plane* 


(21) 
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The  line  through  the  point  (-x(O),  0)  parallel  to  the  y  axis  intersects 
B(X)  at  some  point  T=  [-*(0),  p].  Setting  §  =  T  in  Equation  (21),  we  obtoin 


~  =  h.'  •  (-*(0).  P)  •  (22) 

Recalling  z(X)  ^  ^(t)),  it  follows  from  Equation  (19)  that 

.  x(X)>  V  .  *(T1).  (23) 

From  Equations  (20),  (22),  and  (23),  we  ob^in 

V  .  (-x(O).  p)>X'  .(-X(0).  y°(T)).  (24) 

From  Equation  (24),  it  then  readily  follows  that 

p<y°(T).  (25) 

Solving  Equation  (22)  for  p  yields 

p(X)  =  -X'  •  x(0)  -  g(X),  (26) 


where  we  have  made  the  definition  g(X)  =  •  _z(X)  and  have  introduced  the 

noUtion  p  (X)  to  emphasize  the  fact  that  p'~is  ^function  of  X.  We  note, 
from  Equation  (18),  that 


8(M  =  •  *(>^)  =  /  [x'  •  X"Nt)  B(t)  v(t.  X)  -  ||v(t.  X)|]  <it.(27) 


which  is  clearly  a  function  of  X. 

Now  if  ^(X)  =  £(r|),  we  see  from  Equations  (20)  and  (22)  that 

p(X)  =  y°(T).  (28) 

From  Equation  (25),  we  recall  p(X)  <yO(T)  if  z  (X)^  z(fi).  Thus,  Equation 
(28)  holds  if  and  only  if  _n(X)  =  ^(tj).  ~ 

From  Equations  (25)  and  (28)  then,  we  see  that  the  problem  of  synthe- 
sizing  the  optimal  thrust  program  is  one  of  determining  a  six -dimensional 
vector  X  which  maxindses  the  function  p  (X)  given  in  Equation  (26).  We 
shall  perform  this  maximization  using  the  method  of  steepest  ascent  (Refer¬ 
ence  9). 

The  method  involves  making  X  a  function  of  some  parameter  p  and 
solving  the  differential  equation 
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^  =  kvp(V) 


(29) 


for  X(p).  In  Equation  (29)»  k  is  some  p  instant  and  v  denotes  the 

gradient. 

Let  us  now  consider  the  Uxnit 

X  =  Urn  X(p)  . 
p-^oo 

We  shall  show  that  if  this  limit  exists,  it  is  precisely  a  desired  X  for 
synthesising  the  optimal  control  for  a  given  x(0). 

It  can  be  shown  (Reference  10)  that  7p  (X)  (note  Equations  (26)  and  (27)) 
is  continuous  in  X  and  is  given  by 


Vp(X)  =  -x(0) 


.-1 


(t)B(t)v(t,  X)dt„ 


(30) 


Assuming 

lim  X  (p)  =  X® 
p->cp 

where  X^  is  a  constant  six-dimensional  column  vector,  we  obtain 

lim  7p  [X(p3  =  7p  (X.°), 
p-*oo 

since  7p(X)  is  continuous.  From  Equation  (29)  it  now  follows  that 

Thus,  as  X-^X®,  we  have  dX/dp-^0.  (Otherwise,  X  would  grow  without  bound 
as  p^oo  contradicting  our  assumption  that  X  approaches  a  limit.)  Then, 
for  X  =  X^,  we  see  from  Equation  (30)  that 

-x(0)  =  j  X‘^(t)  B(t)v(t.  \0)  dt.  (31) 

This  means  v(t,  X^)  is  precisely  the  thrust  program  required  to  transfer 
the  space  vehicle  from  the  point  x(0)  to  the  desired  terminal  state  x(t)’  =  0. 
It  is  very  easy  to  show  (Reference  8)  that  £(X®)  s  ^(t|),  and,  therefore,  that 
v(t,  X®)  is  an  optimal  thrust  program. 
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If  we  substitute  Equation  (30)  into  Equation  (29),  we  obtain 


d\ 

a?  = 


-k 


x(0)  + 


X’^t)  B(t)  v(t.  V)dt  . 


or  equivalently, 


r  T 

. 

§  =  -kX-‘(T)< 

X(T) 

x(0)+  1  X'^  (t)  B(t)  v(t.  X)  dt  > 

Jo  J, 

(32) 


We  observe  that  the  term  within  the  braces  is  simply  the  solution  x(t)  of 
Equation  (5)  with  u(t)  =  v  (t,  X)  evaluated  at  the  terminal  time  t=  T.  Since 
this  solution  depends  upon  the  choice  of  X,  we  shall  adopt  the  notation 


x(T,  X)  =  X(T) 


r 

x(0)  + 

X'Nt)  B(t)  v(t,  X)dt 

J 

o 

(33) 


Substituting  Equation  (33)  into  Equation  (32),  we  have 

=  -k  X  (T)  x(T,  X), 

or,  in  discrete  form, 

-  KX'^T)  x(T.  (34) 

where  K  is  some  positive  constant  and  i  is  the  index  of  iteration. 

We  now  outline  the  steps  of  an  iterative  computational  procedure  for 
determining  a  correct  X: 

(1)  Make  an  arbitrary  initial  "guess^'  X^^^  for  X. 

(2)  For  this  value  of  x(^),  compute  v(t,  X(^))  from  Equation  (!?)• 

(3)  Integrate  the  equations  of  motion  (Equation  (5))  using  the  thrust 
program  of  step  (2)  to  dietermihe  x(T,  xt^)).  Equation  (3J)  may  be  used 
directly  for  this  purpose. 

(4)  Substitute  this  value  of  x(T,  x(^))  into  Equation  (34),  solve  for 
x(^+^),  and  return  to  step  (2). 

(5)  Terminate  the  iteration  process  when  |x(^“*‘^)  -  x(^)(|  <  C  where 
e  is  a  small  positive  number. 

The  optimal  thrust  program  is  then  given  by  Equation  (17)  using  the  last 
iteration  on  X. 
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DISCUSSION  OF  RESULTS 


The  utility  of  space  missions  is  directly  dependent  on  their  payload 
capability.  In  minimizing  the  amount  of  fuel  used  in  the  midcourse  phase^ 
our  intent  has  been  to  increase  this  capability,  that  is,  to  permit  the  inclu¬ 
sion  of  additional  useful  equipment.  We  must  bear  in  mind  that  a  sayings  of 
only  a  few  pounds  may  permit  a  mission  to  include  scientific  experiments 
which  would  not  be  possible  otherwise.  This  is  a  significant  factor  when 
viewed  in  terms  of  the  over-all  cost  of  a  space  mission. 

In  developing  the  equations  of  motion  of  a  space  vehicle  in  midcourse 
flight,  we  assumed  the  mass  M  of  the  vehicle  was  appreciably  larger  than 
the  mass  of  fuel  m  consumed  during  the  midcourse  phase.  More  realistic¬ 
ally,  of  course,  we  should  have  considered  the  variations  in  total  mass  along 
the  flight.  However,  if  the  fuel  mass  for  the  midcourse  phase  should  turn  out 
to  represent  a  significant  percentage  of  the  total  vehicle  mass,  this  would 
raise  the  question  of  the  effectiveness  of  the  launch  or  deboost  guidance  sys¬ 
tem.  In  general,  we  would  expect  errors  to  be  small  at  the  conclusion  of 
launch  or  deboost  operations.  The  task  of  midcourse  guidance  then  becomes 
that  of  making  small  corrections  so  that  these  errors  do  not  propagate  along 
the  trajectory  and  cause  large  misses  at  the  destination.  Hence,  for  pur¬ 
poses  of  this  preliminary  study,  it  is  felt  the  assumption  M  >>  m  is  reason¬ 
able. 

We  observe  from  Equation  (17)  that  whether  or  not  the  vehicle  is  thrust¬ 
ing  is  controlled  by  whether  or  not  Jx*  •  X*1  (t)  B(t)|  exceeds  a  threshold 
equal  to  \mity.  Moreover,  we  note  that  when  the  vehicle  is  thrusting,  the 
system  utilizes  the  full  thrust  capability  by  making  |{v(t,  X)||  =  1  and  changes 
only  the  thrust  direction  in  accordance  wi^  the  components  of  X'  •  (t)B  (t). 
As  a  result  of  this  *'on-off'*  nature  of  the  thrust  program,  non-throttleable 
propulsion  can  be  used.  However,  since  the  direction  of  thrust  is  time- 
varying,  the  attitude  control  system  must  be  utilized  during  periods  of  thrust¬ 
ing. 


Since  the  initial  error  x(0)  is  a  measured  quantity,  i.  e.,  the  output  of 
the  navigation  scheme  employed,  the  thrust  program  performance  is  directly 
dependent  upon  the  quality  of  navigation.  By  allowing  the  navigation  system 
more  smoothing  time,  i.e.,  deferring  the  initiation  of  midcourse  guidance, 
one  obtains  a  more  accurate  estimate  of  x(0).  However,  in  waiting,  one 
allows  the  launch  or  deboost  burnout  errors  to  propagate  unchecked.  A  trade¬ 
off  between  these  two  factors  is  essential  in  the  selection  of  the  time  at  which 
midcourse  guidance  should  be  initiated. 

In  conclusion,  we  point  out  that  whether  or  not  the  midcourse  guidance 
scheme  developed  here  is  practical  from  a  systems -hardware  point  of  view 
is  a  question  requiring  further  study.  In  this  section,  we  have  pointed  out  a 
number  of  factors,  pro  and  con.  Certainly,  the  discussion  is  not  exhaustive. 
In  any  event,  it  is  felt  that  results  developed  here  could  be  exploited  as  guide 
lines  in  other  similar  studies.  For  example,  the  amount  of  fuel  required  to 
execute  a  mission  using  the  above  scheme  could  be  used  as  an  ultimate  per-- 
formance  limitation  in  the  design  of  other  midcourse  guidance  systems  for 
the  same  mission. 
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SUMMARY 

A  review  of  the  literature  shows  that  the  problem  of  optimal 
control  synthesis  Is  not  completely  solved  although  many  aspects 
of  the  problem  have  received  considerable  study.  An  Iterative 
procedure  for  the  synthesis  of  time  optimal  controllers  Is  des¬ 
cribed  and,  some  experimental  results  are  presented.  It  is  well 
known  that  the  optimal  control  law  for  the  time  optimal  control 
of  linear  systems  with  bounded  control  amplitude  Is  of  bang-bang 
type  while  the  time  optimal  rocket  steering  control  law  is  con¬ 
tinuous.  In  both  cases,  the  solution  of  the  associated  two-point 
boundary  problem  requires  the  determination  of  (n  -  1)  parameters 
for  an  ntn  order  system.  The  computational  procedure  described 
here  Is  based  on  the  idea  of  optimal  evolution  and  uses  the  method 
of  steepest  descent  to  obtain  the  proper  values  of  the  n  -  1 
parameters.  The  geometrical  significance  of  these  parameters  as 
well  as  their  relationship  to  the  reachable  sets  is  discussed.  An 
experimental  Investigation  of  the  convergence  of  the  iterative 
procedure  Is  described  for  both  regulator  and  controller  problems. 
The  use  of  this  Iterative  technique  for  variable  input  signals  has 
been  found  to  work  in  those  cases  investigated.  The  development 
of  the  theory  shows  that  conV|ergence  to  the  correct  solution  can¬ 
not  always  be  guaranteed  when  working  with  variable  Inputs.  For 
example,  it  Is  possible  for  moving  targets  to  "out  run"  or  "out 
maneuver"  the  system. 

The  results  presented  for  the  bang-bang  problem  were  obtained 
on  second  order  systems.  The  rocket  steering  problems  provided 
third  and  fourth  order  examples.  A  closed  loop  optimal  controller 
for  optimal  rocket  steering  has  been  programmed  and  Is  presently 
being  studied  on  a  digital  computer. 

INTRODUCTION 

A.  A  dynamic  system  is  to  be  controlled  in  such  a  way  as  to 
maximize  or  minimize  some  performance  functional.  The  state  of  the 
system  at  any  Instant  Is  described  by  n-lndependent  variables 
Xi(  t),  1  <  1  <  n.  These  variables  change  with  time  according  to  a 
set  of  di7ferential  equations  Xj^  =  fi(Xjj  Ug,  t)  where  the  Ur  are 
Independent  controls,  1  <  K  <  R  <  n.  These  are  the  governing 
equations  for  the  systeiri.  T?ie  variables  xi  or  may  not  cor¬ 
respond  directly  to  the  physical  variables  naturally  associated 
with  the  process.  In  addltlpn  to  the  governing  equations  there 
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are  constraints  on  the  state  variables  and  on  the  controls.  These 
constraints  are  often  In  the  form  of  Inequalities  A(Xi,  Uj^)  <  0  or 
/c?  B(Xii  Ujf)  <  M.  The  functional  to  be  extremized  is  often  ¥aken 
to  be  an  integi-al  of  the  form  S  =  L(xi^  t)dt  where  the 

function  L  is  called  the  error  criterion  and  the  functional  S  is 
called  alternatively  the  performance  index,  the  payoff  or  the  cost. 

In  realistic  cases  unwanted  disturbances  enter  the  system 
along  with  noise  in  the  sensors  that  observe  the  state  variables  qt 
the  measurable  physical  variables.  The  controller  may  also  suffer 
from  imperfections  of  various  types  and  in  addition  the  controller 
may  have  only  Imperfect  knowledge  of  the  process  or  system  being 
controlled. 


The  initial  arid  desired  terminal  states  of  the  system  are 
prescribed.  The  problem  consists  in  finding  a  control  law  u(t)  in 
the  class  of  allowable  controls  that  brings  the  state  point  from 
initial  to  terminal  locations  and  extremlzes  the  payoff  or  cost. 


A  synthesis  procedure  for  an  optimal  system  results  in  a 
closed  feedback  loop  representing  the  system  being  controlled  and 
a  computer  to  select  the  optimal  control  law.  The  block  diagram 
in  Figure  1  Illustrates  the  main  ideas. 


FIGURE  1.  OPTIMAL  FEEDBACK  CONTROLLER 


It  is  necessary  to  Incorporate  the  noise  and  random  Inputs 
from  the  beginning  and  to  determine  the  extent  to  which  these  dis¬ 
turbances  must  be  taken  into  account  in  selecting  a  performance 
criterion. 

The  synthesis  procedure  involves  several  distinct  steps. 

Each  of  these  steps  is  an  Interesting  area  for  research  on  its 
own  merits.  However,  no  single  step,  by  itself  is  sufficient  to 
constitute  a  synthesis  procedure.  These  steps  Include  the  selec¬ 
tion  of  a  performance  criterion,  determination  of  the  system  state 
point  from  noisy  data,  estlmatlpn  of  statistical  properties  of 
noise  and  random  inputs,  and  computation  of  the  optimal  control  law. 

B.  Historical  background.  A  search  of  the  literature  dis¬ 
closes  many  examples  of  a  specific  nature  as  well  as  several 
thorough  treatments  of  general  methods  of  formulating  problems  of 
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optimal  control.  However,  very  little  has  been  published  regarding 
actual  synthesis  of  real  time  optimal  controllers.  Research  on 
real  time  optimal  control  appears  to  be  confined  In  the  main  to 
separate  studies  of  one  or  more  of  the  Individual  steps  described 
previously.  However,  there  are  some  results  on  synthesis  of  pre¬ 
dictive  but  not  necessarily  optimal  controllers. 

The  reduction  of  the  synthesis  problem  to  studies  of  Individual 
simpler  problems  follows  the  traditional  pattern.  General  studies 
of  dynamical  systems  lead  to  special  studies  of  properties  of  con¬ 
trolled  systems.  The  historical  development  of  control  theory 
usually  emphasizes  stability.  Although  this  Is  always  a  considera¬ 
tion,  It  Is  not  the  main  concern  here.  The  optimization  problem 
Is  frequently  treated  separately  and  ordinarily  is  formulated  in 
terms  of  calculus  of  variations,  either  in  classical  form  or  by 
using  direct  methods.  The  treatment  of  the  optimization  problems, 
in  a  great  many  cases,  is  aimed  at  finding  solutions  for  a  few 
initial  states  and  limited  number  of  terminal  states.  The  two- 
point  boxondary  problem  is  ordinarily  solved  for  only  a  restricted 
class  of  initial  and  end  conditions.  The  solution  of  two-point 
boundaiy  problems  constitutes  another  large  area  of  research  and 
again  there  are  a  number  of  general  studies  and  a  rujmber  of  very 
specific  studies. 

Finally,  there  is  a  class  of  papers  devoted  mainly  to  appli¬ 
cations  of  previously  derived  theoretical  and  computational  results 
to  specific  systems.  These  papers  encompass  aircraft  performance 
studies,  space  travels,  chemical  plant  operation.  Instrument  servo 
designs,  optimal  noise  filters,  and  a  variety  of  other  special  ap¬ 
plications.  It  goes  without  saying  that  there  are  papers  that  fit 
into  more  than  one  of  the  general  categories  described  as  well  as 
articles  of  Interest  that  do  not  fit  well  into  any  one  of  these 
general  areas. 

The  behavior  of  dynamic  systems  subject  to  random  inputs  or 
disturbances  has  some  connection  with  research  on  optimal  control 
ssmthesls.  There  is  a  relation  between  certain  random  problems  of 
control  theory  and  problems  arising  from  physics,  for  example 
Brownian  motion.  The  similarity  between  problems  is  very  close 
and  the  same  kind  of  equation  (Pokker-Planck)  arises  from  both 
classes  of  problems.  Blackman  (l)  Hopkln  and  Wang  (2)  and  Barrett 
(3)  have  studied  this.  The  book  by  Wax  (4)  contains  a  collection 
of  stochastic  problems  of  physics  that  relate  to  control  theory. 
Texts  by  Tsien  (5)  Banning  and  Battln  (6)  and  Petersen  (7)  have 
discussed  random  processes  and  control  theory  in  detail.  In  ad¬ 
dition,  recent  papers  by  Aokl  (8)  Bellman  (9)  Adorno  (lO)  and 
Zadeh  (ll)  have  also  been  concerned  with  control  and  random  proces¬ 
ses. 


The  design  of  optimal  controllers  depends  upon  the  selection 
of  performance  criteria.  Various  kinds  of  standards,  performance 
criteria,  and  error  indices  have  been  proposed  and  Investigated 
(12,13). 
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The  theory  of  optimization  has  developed  along  two  parallel 
branches  stemming  from  mathematical  and  engineering  interests. 
Formulations  of  the  mathematical  problem  have  been  made  by  Bliss 
(14)  Hestenes  (15)  Kalman  (l6)  Marcus  and  Lee  (ly)  Breakwell  (l8) 
Bryson  (I9)  Bellman  f20)  Kalaba  (21)  La  Salle  (22)  Pontriagln  (23) 
Klpiniak  (24)  Chang  (25)  DeSoer  (26)  and  many  others.  A  review  of 
the  literature  relating  to  time  optimal  control  has  been  prepared 
by  Paiewonsky  (27),  At  the  present  time,  the  formulation  of  . 
optimization  problems  and  the  resulting  necessary  conditions  are 
fairly  well  understood.  A  possible  exception  is  the  case  of  in¬ 
equality  constraints  on  the  state  variables. 

Once  the  problem  has  been  formulated  in  precise  mathematical 
terms,  the  next  step  is  to  obtain  solutions.  The  solutions  can  be 
the  optimal  path  that  the  system  should  follow  or  the  optimal  control 
law  to  be  applied  to  the  system. 

The  solution  of  the  optimization  equations  has  been  attempted 
by  many  Investigators  using  several  different  techniques.  For  the 
most  part,  these  investigations  have  been  directed  towards  solu¬ 
tions  of  a  special  class  of  optimization  problems. 

The  direct  integration  of  the  Euler  equations  has  been  tried 
many  times.  Integrations  are  tried  forward,  backwards,  or  in 
combination.  There  are  several  ways  to  satisfy  the  boundary  con¬ 
ditions.  These  Include  (a)  trial  and  error  searching,  for  the 
Lagrange  multipliers,  (b)  flooding  or  calculating  a  great  nvunber 
of  trajectories  by  systematically  varying  multipliers  at  initial 
or  terminal  points,  (c)  guessing  and  refining  the  initial  results 
by  Newton's  method.  Influence  fxmctions  methods  and  other  means. 

It  has  been  known  for  a  long  time  that  the  direct  integration 
of  the  equations  of  the  system  coupled  to  the  Euler  equations  re¬ 
sults  in  a  set  of  equations  with  extreme  sensitivity  to  initial  or 
terminal  conditions. 

Another  approach  to  the  solution  of  the  optimization  equations 
is  the  partial  differential  equation  or  payoff  surface  approach. 

The  Justification  for  these  methods  is  based  on  the  fact  that  for 
most  systems  the  surfaces  of  constant  optimal  payoff  are  boundaries . 
of  convex  sets  and  the  gradient  of  the  surface  (if  it  exists)  cor¬ 
responds  to  the  Lagrange  multipliers  of  the  classical  approach  or 
the  adjoint  variables  of  the  maximum  principle.  These  optimal 
payoff  surfaces  are  solutions  of  a  partial  differential  equation 
analogous  to  the  Hamllton-Jacobl  equation  obtainable  from  the 
maximum  principle.  The  properties  of  these  surfaces,  also  called 
reachable  sets,  have  been  studied  by  Halkin  (28)  Paiewonsky  (27) 
Roxln  (29)  and  Anderson  (30).  The  characteristics  of  this  partial 
differential  equation  satisfy  the  Euler  equations.  A  discrete 
version  of  this  partial  differential  equation  is  the  basis  for  the 
method  of  Dynamic  Programming  developed  by  Bellman.  This  approach 
has  been  applied  to  many  problems  with  varying  degrees  of  success. 

A  technique  based  on  results  of  Neustadt  (3l)  appears  promising 
for  optimal  problems  with  linear  processes.  This  will  be  dls- 
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cussed  in  more  detail  In  a  following  section. 

The  methods  listed  above  have  In  common  the  Idea  of  searching 
for  a  function  that  satisfies  Initial  and  terminal  conditions 
among  a  class  of  functions  satisfying  certain  necessary  conditions. 
An  alternative  appi^oach  has  been  taken  by  several  investigators. 

In  particular,  Kelley  (32)  Bryson  (33)  Swanlund  (34)  Klplnlak  (24) 
Ho  (35)  and  Dreyfus  (36)  have  Investigated  the  gradient  methods. 
These  methods  seek  a  control  function  that  extremlzes  the  perfor¬ 
mance  functional  from  the  class  of  functions  that  satisfy  the 
boundary  conditions.  The  method  depends  upon  successive  approxima¬ 
tions  to  continually  improve  the  value  of  the  performance  Index. 

In  the  course  of  the  "descent"  It  Is  necessary  to  adjust  the  pro¬ 
cedure  to  satisfy  the  boundary  conditions  which  may  be  drifting 
off.  'Several  special  procedures  developed  to  do  this  have  been 
reported  on  and  are  included  in  the  bibliography.  Analog  computers 
for  solving  these  problems  have  been  investigated  by  Klplnlak  (24) 
and  Meers  (37 ) . 

The  optimal  control  of  quantized  or  discrete  systems  has  been 
studied  by  DeSoer  (38)  Prledland  (39)  Kullkowski  (4o)  Neustadt  (4l) 
and  Zadeh  (ll)  among  others. 

The  Idea  of  a  field  of  extremals  is  fairly  old,  however,  it 
has  been  only  recently  that  this  notion  has  been  applied  to  optimal 
control  problems  to  obtain  neighboring  solutions  to  a  known  optimal 
trajectory. 

Perturbation  techniques  designed  to  obtain  additional  solu¬ 
tions  to  boundary  problems  by  examining  neighboring  solutions  have 
been  studied  by  Bryson  (42)  Kelley  (43)  Klplnlak  (24)  Swanlund  (34) 
and  Dow  (44)  among  others. 

There  Is  such  an  abundant  literature  on  applications  of  opti¬ 
mization  procedures  to  problems  In  flight  mechanics  that  It  Is 
impractical  to  do  more  here  than  to  cite  some  of  the  more  prominent 
works  and  to  describe  in  generalities  what  has  been  accomplished. 

It  is  Important  to  observe  that  the  objective  In  many  of  these 
studies  Is  to  find  only  a  few  solutions  to  a  limited  class  of 
boundary  conditions  and  the  question  of  synthesis  of  a  controller 
does  not  arise . 

Classical  variational  theory  has  been  used  by  Leltmann  (45) 
Behrbohm  (46)  Mlele  (47)  Melbourne  (48)  and  Edelbaivn  (49)  among 
others  to  Study  optimal  paths  for  aircraft,  rockets,  and  space 
craft..  Lawden  (50)  employed  direct  methods  for  optimal  orbital 
transfers,  while  Bellman  (5l)  and  Dreyfus  (52)  and  Smith  (53)  use 
dynamic  programming. 

Optimal  reentry  and  optimal  climbs  have  been  among  the  ap¬ 
plications  of  the  gradient  methods  employed  by  Bryson  (54)  and 
Kelley  (55) •  These  Investigators  have  also  given  some  considera^ 
tlon  to  the  synthesis  problem.  Frledland  (56)  has  considered  the 
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synthesis  problem  and  has  described  in  general  terms  the  structure 
of  a  class  of  optimal  controllers. 

At  this  point  It  Is  useful  to  look  back  for  a  moment  and  take 
stock  of  what  has  been  said .  Optimal  control  and  optimal  path 
problems  can  be  formulated  and  solved  for  a.  number  of  special  ap¬ 
plications.  In  general,  though,  these  are  idealized  cases.  That 
Is,  given  a  typical  problem  as  posed  In  the  preceding  sections.  It 
Is  usually  possible  to  find  a  solution  to  the  two-point  boundary 
problem,  if  one  exists,  by  some  one  of  a  variety  of  means.  Never¬ 
theless,  this  is  not  enoiigh  to  constitute  a  synthesis  procedure, 
although  it  is  a  step  in  the  right  direction. 

A  closed  loop  optimal  controller  must  be  able  to  determine  the 
optimal  control  law  at  each  Instant  of  time.  Except  for  some 
special  cases,  l.e,  time  invariant  linear  processes  with  quadratic 
error  and  quadratic  fuel  cost,  the  two-point  boundary  problem  re¬ 
quires  Iterative  solution  in  one  form  or  another. 

The  determination  of  the  optimal  control  may  be  accomplished 
by  looking  up  stored  results  of  previous  calculations,  by  actual 
solution  of  the  two-point  boundary  problem  with  fast-time-scale 
computers,  or  by  a  combination  of  these  methods.  For  example, 
controllers  have  been  described  by  Bryson  (57)  and  Kelley  (43) 
that  would  use  stored  nominal  optimal  paths  together  with  a  fast 
computer  to  determine  neighboring  optimal  paths.  Alternatively, 
nominal  paths  can  be  stored  and  the  optimal  control  for  an  "Initial" 
point  not  on  a  stored  path  can  be  approximated  by  an  Interpolation 
scheme . 

There  are  many  alternatives  available  and  only  a  few  seem  to 
have  received  any  serious  study.  For  the  most  part,  these  studies 
are  idealized  by  excluding  noise,  random  disturbances,  and 
component  Imperfections.  There  are  exceptions  however.  Studies 
of  satellite  attitude  controllers  have  included  these  effects  and 
considered,  the  behavior  of  the  systems,  (usually  relay  type),  in 
the  small  signal  region  with  component  imperfections  present  such 
as  dead  time  and  hysteresis  (58).  The  effect  of  these  real 
characteristics  on  limit  cycle  size  and  fuel  consumption  are  es¬ 
pecially  important  in  those  applications  Involving  a  service  life¬ 
time  that  is  long  compared  to  system  time  constants.  An  optimal 
system  designed,  for  example  to  reduce  large  errors  in  least  time, 
should  be  expected  to  spend  most  of  its  life  in  the  small  signal 
region  where  the  effects  of  imperfections  and  noise  are  important. 

Returning  again  to  the  synthesis  question.  It  has  been  found 
worthwhile  to  examine  some  results  of  studies  of  non-optimal  con¬ 
trollers  that  use  techniques  similar  in  principle  to  those  re¬ 
quired  for  optimal  synthesis.  These  are  the  predictive  controllers 
and  the  terminal  controllers.  Some  results  of  automatic  predictive 
controllers  are  available.  (Cheshut  and  Solllclto  (59)  Steeg  and 
Morris  (6o).)  The  applications  studied  Include  reentry  guidance, 
satellite  rendezvous  and  airplane  automatic  landings.  In  ad^ 
dltion,  studies  of  predictive  controllers  with  human  pilots  in  the 
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loop  have  also  been  made  (NASA)  (6l,62).  These  have  also  been 
Idealized  problems  as  various  simplifications  or  approximations 
have  been  made  to  facilitate  the  studies. 

These  predictive  controllers  have  been  made  to  work  at  least 
In  Ideal  simulation  and  there  seems  to  be  no  fundamental  reason 
why  similar  controllers  based  on  an  optimal  control  logic  could 
not  be  made  to  work  under  the  same  Ideal  conditions.  The  Important 
question  Is  how  to  design  an  optimal  controller  to  operate  under 
real  conditions  and  very  little  Is  known  about  this  at  present. 

The  next  section  will  describe  some  results  of  optimal  synthe¬ 
sis  studies  for  linear  systems. 


SYNTHESIS  OF  OPTIMAL  CONTROLLERS  FOR  LINEAR  SYSTEMS 

A.  Preliminary  remarks.  The  discussion  In  the  preceding  sec¬ 
tion  defined  an  optimal  controller  as  a  closed-loop  feedback  system 
that  repetitively  solves  the  two-point  boundary  problem.  It  was 
pointed  out  that  there  are  several  ways  to  accomplish  this  de¬ 
pending  on  the  amount  of  pre -computation  and  storage  allowed.  The 
studies  reported  on  here  use  the  minimum  amount  of  precomputa¬ 
tion.  The  minimum  time  performance  criterion  was  selected  for 
these  synthesis  studies  for  several  reasons.  The  main  reason  being 
the  belief  that  a  synthesis  procedure  could  be  achieved  for  this 
case.  The  reachable  sets  are  generally  easy  to  obtain,  yet  the 
problems  are  not  trivial.  The  next  step  was  to  try  to  obtain  a 
computational  solution  to  the  time  optimal  control  problem.  The 
closed-loop  system  that  was  envisioned  would  frequently  sample  the 
measurable  system  output  variables  and  estimate  the  system  state 
point.  On  the  basis  of  this  estimate,  the  controller  rapidly  ob¬ 
tains  the  optimal  control  manipulation  by  finding  the  proper  state 
for  the  adjoint  system.  Section  B  contains  a  description  of  the 
studies  directed  towards  developing  a  useable  computational  pro¬ 
cedure.  The  Incorporation  of  this  procedure  Into  a  feedback 
controller  Is  described  In  section  C.  Some  applications  to  simple 
problems  of  flight  mechanics  were  used  to  provide  a  framework  for 
the  synthesis  studies. 

B.  Solution  of  the  time  optimal  control  problem.  It  Is  al¬ 
ready  well  known  that  the  solution  for  the  classical  time  optimal 
control  problem  Is  of  bang-bang  type  and  the  control  law  can  be 
given  In  terms  of  the  response  of  the  system  adjoint  to  the  given 
system.  It  Is  possible  to  obtain  these  results  by  several  different 
procedures  and  a  review  of  the  field  Is  contained  In  reference  (27). 

In  spite  of  the  fact  that  the  optimal  control  law  has  been 
known  for  a  long  time,  very  few  optimal  controllers  have  been  built 
for  systems  of  order  higher  than  two.  The  principal,  drawback  has 
been  the  difficulty  In  solving  the  two-point  boundary  problem,  or 
what  Is  the  same  thing,  finding,  the  optimal  Initial  state  of  the 
adjoint.  This  section  Is  Intended  primarily  to  show  hpw  previously 
published  techniques  can  be  combined  to  successfully  solve  this 
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problem.  In  addltlonj  some  experimental  results  are  presented  that 
provide  verification  of  the  convergence  of  the  proposed  procedure 
for  a  class  of  linear  time  Invariant  systems.  The  problem  can  be 
stated  as  follows:  Given  a  linear  system  of  the  form 

X  =  A(t)X  +  B(t)u  (l) 

where 


X  Is  an  n-vector 
A  Is  n  X  n  matrix 
B  is  n  X  r  matrix 
u  Is  r  vector,. 

It  is  desired  to  find  the  control  law  u°  that  takes  the  system  from 
Initial  state  X(0)  to  terminal  state  X«(or  catches  Input  signal 
i{t)  in  least  time,  T°. 

It  Is  well  known  that  the  optimal.. control  law  Is  given  by  the 
expression 

u°  =  sgn  (t]  •  X“^B)  (2) 

where  X~^(t)  satisfies  the  adjoint  differential  equation 

(r^)  =  -(x'^)A 

The  solution  of  the  equation  (l)  can  be  written  as 


pT 

x(t)  =  X(t)[x(o)  +  J  X  (t)  B(T)u(T)dT  I 


(3) 


After  s-'ubs'titutlng  the  terminal  condition  x^(t)  =  ?j^(t)  and  equa¬ 
tion  (2)  the  following  expression 


-x(o)  =  X‘^(t)4(t)  +  B(T)sgn 


tiX"^(t)B(t)^t  (4) 


is  obtained. 

This  equation  in  effect  Is  a  mapping  of  vectors  -q  into  vectors 

z  =  -  r  ^ . . .  dT  c(t)  =  0 . 

The  problem  Is  to  find  an  t]  that  maps  Z  Into  -x(o).  This  was 
recognized  In  a  somewhat  different  form  by  Krasovskil  (63) .  How¬ 
ever,  Neustadt  provided  a  key  to  a  steepest  descent  procedure  by 
means  of  an  elegant  result  (31). 


Before  proceeding  directly  to  the  main  idea.  It  is  desirable 
to  insert  a  brief  discussion  of  .some  geometrical  aspects  of  the 
problem,.  The,  idea  of  optimal  isochrones,  or  the,  boundaries  of  the 
reachable  sets,,,  plays  an.  Important  role  in  the  development  of 
optimal  control  theory.  To  each  point  in  the  x-phase  space,  there 
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Is  assigned  a  value  of  the  optimal  time  required  to  reach  the  target. 
Through  each  point  there  passes  a  portion  of  a  surface  of  constant 
optimal  time  to  reach  the  target  (Figure  2).  The  normal  to  this 
surface  defines  the  direction  of  the  optimal  t)  vector.  These  sur¬ 
faces  need  not  be  smooth.  In  fact,  there  may  be  corners,  sharp  . 
edges  and  points.  Therefore,  the  gradient  may  not  exist  at  certain 
points  or  along  certain  curves  or  hypersurfaces.  Figure  3  shows  a 
typical  set  of  Isochrones  for  a  second-order  system. 

The  hyperplane  passing  through  the  point  -x(o)  and  defined  by 
the  vector  Is  a  support  plane  for  the  convex  set  formed  by  the 
optimal  isochrones.  Therefore,  for  all  unit  vectors  t],  the  Inner- 
product  (-n,  -x(o))  <  (t)?  -x(o))  or  t)®  maximizes  (tj^  -x,(o))  . 

Neustadt  (31 )  suggests  the  following  iteration  procedure.  Select 
a  trial  value  of  t].  This  determines  a  trial  hyperplane  at  -(x(o)) 

In  the  Z-space.  Examine  the  locus  of  points  mapped  by  t)  into  Z 
using  t  as  a  parameter,  and  select  the  values  of  Z  and  t  correspond¬ 
ing  to  the  point  where  the  locus  pierces  the  hyperplane.  The 
geometrical  Ideas  relating,  the  optimal  control  problem  to  the 
notion  of  expanding  convex  sets  are  believed  to  have  been  first 
explored  by  Bellman,  Gllcksberg,  and  Gross  in  a  study  of  the  bang- 
bang  problem  (20). 

Figure  4  Illustrates  this  procedure.  The  point  of  Inter¬ 
section  of  the  Z  trajectory  and  the  hyperplane  will  not  generally 
coincide  with  the  desired  target;  -X4(o).  If  this  point  does 
correspond  to  -Xj^(o)  then  the  optimal  initial  conditions  of  the 
adjoint  have  been  found  and  are  equal  to  the  components  of  the 
normal  to  the  hyperplane.  A  series  of  corrections  to  the  Tj-vector 
is  usually  necessary  to  find  the  optimal  initial  conditions.  A 
_steepest  descent  procedure  is  effective.  The  changes  in 
1-j^^V+l  ==  T|V  ^  given  by  a  result  of  Neustadt,  namely 

i5t]  =  K[x(o)  +  zfjYyt)].  _The  correction  to  t]'''  should  lie  along  the 
"error  vector"  (x(o)  -t-'Z('n,t))  In  the  hyperplane  determined  by  . 


A  starting  value  for  the  "q  vector  must  be  obtained.  The  re¬ 
sults  of  previous  studies  have  shown  that  there  Is  a  relation 
between  the  optimal  Isochrones  and  Liapunov  functions  (27).  This 
observation  leads  to  a  means  of  approximating  the  optimal  iso¬ 
chrones  by  quadratic  forms.  These  quadratic  forms  are  then  used 
to  obtain  the  starting  values  for  the  iteration.  The'se  studies 
used 


T)(o)  =  - 


\  x(o)  II 


The  Initial  studies  were  based  on  simple  systems.  Using  the  ARAP 
analog-digital  computer,  a  second  order  undamped  oscillatory 
system  with  one  control  force  was  simulated  and  regulated  In  a 
time  optimal  fashion. 


The  equations  are: 
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"  i 

i=l 


X2  = 

2 

-03  +  U 

(1') 

u°  =  sgn[-q^ 

sin  cot  ,1 

— - -  +  cos  a)tj 

(2') 

Z  (t)  =  - 

f  i  sin  coT[u]dT 

(4a') 

Z2(t)  = 

f  COS  C0T[u]dT 

'  0 

(4b') 

+  x^(o)] 

+  q2[Z2(t)  +  x^Co)]  <  0 

(5) 

q  -K  [Z  (T)  + 

n-1  ^ 

x^(o)]  -K2[Z^(t)  +  x^(o)]2 

-  (6a) 

^  II  TJ^  -K3^[Z^(t)  +  x^(o)]  -  K^lZ^iT)  +  x^(o)]‘ 

t  T  rL“  1 


The  analog  computer  is  started  with  a  first  guess  of  the 
adjoint  system  initial  conditions  t]2 

x^(o) 

M  x^(o)  M 

This  initial  guess  is  obtained  from  a  Liapunov  function.  The 
computer  then  runs  until  T]j_[Zi(t)  +  Xi(o)]  becomes  positive.  The 
analog  machine  is  then  transferred  into  the  hold  mode;  Z^,  Zp 
sampled;  and  a  new  initial  condition  vector  111,,  rj2  found  for  the 
adjoint  system.  This  cycle  is  repeated  until  the  terminal  point 
in  Z-space  is  within  distance  e  of  -xl(o). 

For  the  examples  shown  here,  the  system  parameters  are: 
cd2  =  .1  and  F  =  +  .5j  giving  steady  state  rest  points  of  xj  =  +  5* 
In  all  tests,  the  effects  of  the  gain  constants  K]^,  K2  in  t6a)  on 
the  convergence  characteristics  were  noted.  The  general  trend  ob¬ 
served  was  that  increasing  the  K's  resulted  in  fewer  Iterations 
required  per  solution  with  five  iterations  being  the  minimum 
number.  Further  increases  then  resulted  in  large  and  undesirable 
oscillations  about  the  correct  solution.  The  values  of  the  gain 
constants  at  which  the  cycling  occurs  depends  upon  the  system  in 
question  and  its  initial  conditions.  For  the  time  optimal  regula¬ 
tion  of  the  harmonic  oscillator  values  of  =  1;  K2  =  1  were 
satisfactory  under  all  conditions. 

Using  these  nominal  values,  the  iterations  shown  in  Figures  5 
and  6  were  obtained.  Each  of  the  trajectories  shown  in  the  Z-phase 
space  represent  one  of  the  iterations  in  the  search  for  the  correet 
initial  conditions  for  the  adjoint  system.  The  trajectory  of  the 
physical  system^  in  x-phase  space  corresponding  to  the  correct  q 
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vector  is  also  shown.  Figure  6  illustrates  the  same  data  in  time 
history  form. 

In,  addition  to  oscillators,  the  regulation  of  double  integra¬ 
tors  was  also  investigated  =  o).  A  sample  case  is  shown  in 
Figure  7*  The  hyperplane  is  almost  parallel  to  the  trajectory  of 
the  system  in  the  region  of  the  solution.  Under  these  conditions, 
a  small  angular  change  in  the  adjoint  initial  vector,  (producing 
an  equal  angular  change  in  the  trial  tangent  line)  will  result  in 
a  large  displacement  of  the  point  of  intersection.  It  is  necessary 
to  reduce  the  steepest  descent  gains  If  oscillations  are  to  be 
avoided.  In  the  case  shown,  Kq  =  .1,  Kg  -  •!  were  used  in  the 
region  sufficiently  near  to  the  correct  solution  (use  of  these 
gains  throughout  the  problem  results  in  an  excessive  number  of 
iterations).  It  will  generally  be  necessary  to  program  controller 
gain  variations. 

In  addition  to  regulation  (returning  the  system  to  its  null 
point)  it  was  decided  to  investigate  the  capabilities  of  this 
technique  in  following  command  signals.  Referring  back  to  Equa¬ 
tion  4,  it  can  be  seen  that  the  ability  to  follow  variable  input 
signals  is  obtained  by  retaining  the  terms  §4_(t);  l.e.,  the  target 
point  coordinates,  ^i(t),  are  no  longer  located  at  the  origin. 

For  the  second-order  example,  the  equations  used  by  the  controller 
are 

X^(t)  =  x^(o)  -  f  ^  sin  oyrEuldT  -4q(t)  cos  (ut  + 

t 

k2(t)  .=  X2(o)  +  j  cos  ayr[u]dT  -^^(t)cD  sin  cot  cos  ot 

(7b ) 

where  Xj^(t)  =  xq(o)  +  Zq(t)  terms  resulting  from  ?q(t)  /  o. 

tJ3^[X^(t)]  +  ^  ° _ 

Ri  -K  [X  (T)]  -K2[^i(T)]^ 

n-1 _ ; _ 

II  T).  -K  [\  (T)]  -K2[X  (T)]:^  I  I 

These  equations  reduce  to  those  given  previously  by  setting 
^^(t)  =  ?2(^)  ~  ^  transposing  the  Xj^(O)  terms  to  the  left  hand 

side  of  the  equations . 

The  two  forms  of  input  signals  investigated  in  this  portion 
of  the  study  were : 

(a)  Arbitrary  stationary  points  in  the  phase  space,  i.e. 
going  to  a  prescribed  position  and  attaining  a  prescribed  velocity 


(5') 

(8) 
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at  that  point. 

(b)  Step  and  ramp  signals.  In  phase  space,  the  target  starts 
with  initial  horizontal  and  vertical  displacements  from  the  origin 
and  moves  horizontally  at  a  rate  proportional  to  its  vertical 
displacement.  Figure  8  shows  the  iteration  process  for  the  first 
type  of  input. 

It  is  Important  to  remember  that  Neustadt's  procedure  is 
based  on  finding  the  maximum  time  for  which  the  condition 
Tj  •  [Z(t)  +  x(0)]  =  0  can  occur.  The  error  vector,  [Z(t)  +  x(0)] 
need  not  decrease  at  each  step  of  the  steepest  descent  and  may  be¬ 
come  very  large  even  as  the  time  T  approaches  the  optimal  time. 

In  order  to  speed  up  the  convergence  of  the  iterations,  a  change 
is  made  in  the  procedure  if  successive  changes  in  T  are  observed 
to  be  growing  small  without  reduction  In  the  size  of  Z(T)  +  x(0). 
There  are  several  other  ways  to  use  the  Z-equations  and  two  of 
these  have  been  employed  during  the  terminal  phase  of  the  itera¬ 
tions.  Gamkrelldze  suggests  a  method  (also  described  by  Kras¬ 
ovski!  )  for  a  steepest  descent  procedure,  keeping  T  fixed  and 
varying  t]  to  make  Z(T)  parallel  to  x(o).  The  final  time  T  is  to  be  in¬ 
creased  from  something  small  until  Z(T)  =  -x(o).  In  this  method,  as  in 
the  previous  one,  there  is  still  the  question  of  determining  the  maximum 
T  (i.e.  distinguishing  local  minima)  in  the  case  of  moving  targets. 

It  is  also  possible  to  seek  the  value  of  q  that  minimizes 
the  error  at  the  time  q  •  [Z(t)  +  x(0)]  =  0.  This  is  a  modifi¬ 
cation  of  a  search  method  Investigated  by  Paiewonsky  (27 ) .  The 
latter  method  usually  requires  an  experimental  determination  of 
the  gradient  in  practice,  although  a  closed  form  solution  is 
sometimes  available.  The  results  obtained  to  date  on  these  tech¬ 
niques  are  not  sufficient  to  allow  a  complete  comparison  with 
Neustadt's  method  regarding  speed  of  convergence,  practical  dif¬ 
ficulties,  etc.  It  is  hoped  that  it  will  be  possible  to  do  this 
soon.  It  is  possible,  however,  to  state  that  the  problem- of  the 
vanishing  of  the  gradient  of  the  error  (as  a  function  of  the 
initial  q)  as  described  by  Paiewonsky  in  reference  (27)  can  occur 
for  the  latter  method..  This  does  not  occur  with  Neustadt's 
method  as  the  gradient  ther4.~^ls  always  given  by  the  error  vector. 

C.  Applications.  The  applicability  of  this  technique  to 
problems  of  flight  mechanics  depends  upon  a  linearization  of 
some  kind.  Many  possibilities  suggest  themselves;  one  is  to 
find  a  problem  that  can  be  linearized  from  the  start.  If  this 
cannot  be  done,  it  may  be  possible  to  apply  a  method  of  suc¬ 
cessive  approximations  to  the  equations  solving  a  succession  of 
linear  problems.  The  following  example  was  prepared  in  order 
to  study  the  properties  of  a  closed  loop  optimal  controller  using 
a  repetitive  computer.  Consider  first  the  problem  of  determin¬ 
ing  the  optimal  steering  program  for  the  upper  stages  of  a 
boo.ster  rocket.  If  theTi^ocket  burns  continuously  then  the 
mlnim'um  fuel  problem  is  a^  minlmiim  time  problem.  The  gravitational 
field  will  be  assumed  to  be  uniform  in  this  example.  The  equa- 
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tlons  of  motion  are 


Xn 

J. 

-  X2 

^2 

=  A  cos 

u 

= 

=  A  sin 

u  -  g 

(9) 


The  convexity  of  the  reachable  sets  can  be  verified  directly. 
The  adjoint  equations  are : 


(10) 


The  optimal  control  is  the  well-known  bl-linear  tangent  law: 


^  P^(t)  P^(0)  -  P^CO)! 

-  P^TBJ. "  P2(0>  -  P3,(0)t 


In  this  example,  the  equations  for  the  Z's  are 

=  -  t  A  cos  u 
Z„  =  A  cos  u 

.  (11) 

Z^  =  -  t(A  sin  u  -  g) 

Z^i^  =  A  sin  u  -  g 

The  particular  numerical  example  chosen  is  the  optimal 
steering  of  a  nuclear  powered  upper  stage  rocket.  The  initial 
conditions  are 

Yjj(o)  =  10,330  ft/sec 
VyCo)  =  5,500  ft/sec 

y(0)  =  20  miles 

and  the  desired  terminal  conditions  are: 

Vjj.  =  25,300  ft/sec 
=  0  ft/sec 
y  =  200  miles 

The  horizontal  range  was  not  prescribed.  These  equations  were 
set  up  on  the  ARAP  analog ^digital  computer  at  first  for  the 
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special  case  of  A  =  P/m(t)  =  5.  The  objectives  of  the  Initial 
study  were  to  determine  the  sensitivity  of  the  convergence  of 
the  Iteration  procedure  to  computational  errors  and  to  determine 
the  number  of  Iterations  required  as  a  function  of  the  Initial 
starting  value  of  the  adjoint.  These  studies  showed  that  con¬ 
vergence  of  the  Iterations  depended  to  a  great  extent  on  the  ac¬ 
curacy  with  which  the  stopping  time  could  be  determined.  For 
systems  of  order  higher  than  two,  Z(T,t))  proved  to  be  a  highly 
sensitive  function  of  tj.  The  maxlmwr-tolerable  er,ror  In 
determining  the  time  at  which  t)  •  (Z(T,'n)  r  x(t)))  vanished  was 
of  the  order  of  10-^  seconds.  This  Is  Illustrated  by  Table  1, 
which  represents  a  typical  case. 


Components  of  t) 

p(n) 

;  Component  s  of  Z  (f(ii  ),Ti)r  x(o) 

.54580027  .01316405  1.0729829 
.54580027  .01316405  1.0729839 
.54580027  .01316405  1.0734830 
.54580027  .01314405  1.0734830 

148.56863 
148.56906 
148.56824 
148 . 56763 

.830  22,700  -7.373 

.397  52,705  -16.369 

1.259  -7,410  1.640 

10.711  -121,610  31.768 

TABLE  I 


This  extreme  sensitivity  makes  It  difficult  to  reduce  the 
error,  |  |z(T,‘n)  -x(o)l  j  when  the  time  T  is  close  to  the  optimal 
time.  Two  steps  were  taken  to  overcome  this  difficulty.  First, 
a  special  program  for  the  step  size  was  developed.  Second,  when 
Successive  values  of  stopping  time,.  T,  differed  by  a  pre¬ 
selected  amount  of  different  computational  scheme  was  used  to 
obtain  reduced  values  of  the  error  |  |z(T,tj)  -  x(o)  |;  I  and  precise 
estimates  of  t|*.  The  details  of  these  computational  studies 
-  wli.'L  be  contained  In  a  forthcoming  ASD  report.. 

The  next  step  was  to  program  a  computational  model  of 
closed-loop  optimal  controller  for  simulation  on  an  IBM  7090* 

This  Is  shown  schematically  In  fig.  9-  The  controller  samples 
the  output  of  the  system  and  determines  the  value  of  the  control, 
uO(t)  on  the  basis  of  the  noisy  measurements.  The  possibility 
of  filtering  the  noisy  measurements  was  Included. 

It  takes  a  finite  amoxint  of  time  for  the  computer  to 
determine  the  control  and  a  variable  length  delay  was  Incor¬ 
porated  to  study  the  effect  of  this  time  lag.  Provision  was 
also  made  for  a  calculation  in  the  controller  to  simulate 
compensation  for  the  delay.  The  control  computer  requires  an 
estimate  of  the  system  parameters.  In  this  case  the  Initial  mass 
m(o),  the  propellant  mass  flow  rate  p,  and  the  thrust  F  or  the 
effective  exhaust  velocity  c.  The  control  computer  does  not 
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know  the  exact  values  of  these  parameters  and  a  study  was  made 
to  determine  the  effect  of  uncertainties  In  system  parameters  i 
on  overall  performance.  The  mass  flow  rate,  p,  was  allowed  to 
have  random  variations  corresponding  to  rough  burning,  or  In 
extreme  cases,  a  "chamber  out"  condition.  No  adaptive  features 
were  Included  In  the  controller  although  It  Is  clear  that  the 
system  parameters  needed  are  capable  of  being  estimated  In  flight. 

In  order  to  reduce  the  niamber  of  Iterations  required  during 
the  computation  of  the  control,  a  "nominal"  trajectory  was 
Introduced.  The  nominal  trajectory  used  was  the  optimal  path 
from  the  Initial  point  to  the;  terminal  point.  That  Is,  once  the 
optimal  Initial  values  for  the  adjoint  are  found  by  the  computer, 
after  the  very  first  sample  period,  the  nominal  values  for  the 
state  variables  and  adjoint  can  be  obtained  by  a  faster  than 
real  time  Integration.  These  values  of  the  adjoint  are  stored 
and  used  as  starting  values  for  the  subsequent  Iterations  at 
later  sampling  periods. 

This  work  Is  not  yet  complete.  However,  some  preliminary 
results  of  the  computer  studies  are  available.  The  objective 
In  the  perturbation  studies  Is  to  observe  the  behavior  of  the 
Iteration  scheme  as  the  Initial  conditions  are  perturbed  away 
from  the  starting  values.  Two  types  of  Investigations  were 
made.  In  a  typical  run  of  the  first  series,  the  Initial  point 
in  phase  space  Is  fixed,  the  starting  value  of  the  adjoint  Is 
varied  and  the  number  of  Iterations  required  to  converge  to  the 
optimal  value  of  the  adjoint  Is  obtained.  The  second  series  of 
runs  Is  similar  but  the  starting  value  of  the  adjoint  is  fixed 
and  the  Initial  state  of  the  system  is  varied.  The  nimiber  of 
iterations  required  to  find  the  optimal  state  of  the  adjoint  Is 
obtained  here  also. 

A  series  of  runs  to  observe  the  closed  loop  system  be¬ 
havior  In  the  presence  of  noise  and  parameter  uncertainties  Is 
also  being  carried  out  at  the  present  time  as  part  of  the 
computer  studies.  The  results  of  these  studies  will  be  re¬ 
ported  In  detail  at  a  later  date . 
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Fi^re  2.  Surface  of  Constant  Optimal  Time 
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Figure  3.  Optimal  Isochrones, 
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Figure  5.  Phase  Diagram  -  Time  Optimal  Regulation 
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Figure  6,  Time  History  of  Optimai  Regulation 
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FIGURE  7.  PHASE  DIAGRAM  TIME 
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ABSTRACT 

Section  I  considers  the  practical  problem  of  closed-loop 
control  of  linear  plants.  A  minimum  response  time  criterion 
is  used  in  the  design  of  a  pitch  attitude  contreller  for  a 
flexible  launch  vehicle.  The  criterion  Is  applied  to  a  fourth 
order  model  containing  the  primary  dynamics  of  the  thirteenth 
order  vehicle.  There  are  tried  and  proven  methods  for 
obtaining  the  control  variable  as  a  function  of  time  v/hich 
takes  the  system  from  an  initial  condition  to  the  target  set 
in  minimum  time.  It  is  shoivn  that  a  suitable  collection  of 
these  open-loop  trajectories  can  be  used  to  define  a  closed- 
loop  control  law.  Results  of  an  analog  simulation  are  pre¬ 
sented  which  show  that  this  control  law  properly  applied  to 
the  flexible  vehicle  results  in  good  control. 

Section  II  considers  finding  the  optimum  path  from  an 
Initial  condition  to  a  target  set.  The  problem  is  reduced 
to  an  initial  value  problem  in  v/hich  the  minimizing  Initial 
values  of  the  adjoint  variables  or  multipliers  are  sought. 
(Inequality  constraints  are  Included  in  the  formulation.) 

The  problem  then  becomes  one  of  minimizing  a  function  of 
several  variables  subject  to  constraint  equations  in  those 
variables.  This  is  a  problem  for  which  necessary  and  suf¬ 
ficient  conditions  for  a  strong  relative  minimum  are  well 
known,  A  second  order  Newton-Raphson  iteration  procedure  for 
numerically  finding  the  minimum  is  described.  Finally, 
experiences  in  the  use  of  the  Newton-Raphson  method:  are  des¬ 
cribed. 
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SECTION  I 


AN  APPLICATION  OF  TBIE- OPTIMAL 
CONTROL  THEORY  TO  LAUNCH  VEHICLE  REGULATION* 

In  spite  of  the  relatively  large  effort  which  has  gone 
Into  the  study  of  optimization  during  the  last  few  years,  there 
have  been  few  applications  to  significant,  practical  closed- 
loop  control  problems.  This  Is  true  In  spite  of  the  fact  that 
theoretical  developments  promise  solutions  or  potential  solu¬ 
tions  to  control  problems  for  which  conventional  synthesis 
procedures  are  not  completely  satisfactory.  Among  the  diffi¬ 
culties  which  have  hindered  practical  applications  are:  ade¬ 
quate  description  of  real  plants  often  requires  differential 
equations  of  quite  high  order,  the  control  law  Is  usually  a 
non-llnear  fiinctlon  of  many  variables  and  difficult  to  Imple¬ 
ment,  and  the  fact  that  the  theoretical  solution  of  the  optimi¬ 
zation  problem  most  often  yields  the  open-loop  control  law 
u  (t,  x(o))  rather  than  the  required  closed-loop  law,  u(x). 

The  flexible  launch  vehicle  Is  used  In  this  paper  to  Illustrate 
these  problems  and  to  demonstrate  the  use  of  some  techniques  to 
overcome  them. 

1.1  Equations  of  Motion  The  assumed  equations  of  a  typical 
250,000  pound  flexible  launch  vehicle  are  given  In  Table  1. 
Poles  and  zeroes  of  the  £r  transfer  function  are  listed  In 

u 

Table  2.  Airframe  coefficients  are  taken  at  the  maxlmim  dyna¬ 
mic  pressure  flight  condition  with  flight  speed  assumed  con¬ 
stant.  The  equations  Include  dyneimlcs  of  the  rigid  body, 
three  body  flexure  modes,  tall -wags  dog,  actxiator,  rate  servo 
and  an  Integration  of  pitch  rate  for  control  of  pitch  attitude . 
A  single  control  variable  Is  assxamed  ava_lable  from  glmbaling 
of  the  engine.  A  maxlmxim  glmbal  rate  of  0.2  rad/sec  Is  com¬ 
manded  at  all  times. 

1.2  Specification  of  the  Controller  In  applying  optimal  con¬ 
trol  theory  to  the  synthesis  of  controllers  for  practical 
plants  It  Is  necessary  to  specify  both  the  optimization  cri¬ 
terion  and  what  Is  to  be  controlled.  With  the  criterion  used 
here,  minimum  response  time.  It  has  been  common  to  apply  the 
criterion  to  the  state  vector  y,  of  a  plant  In  the  form, 

(1.2.1)  y  =  Ay  +  Bu 

However,  when  this  Is  done  the  resuating  response  In  multi¬ 
degree  of  freedom  systems  may  be  entirely  imacceptable .  This 
is  forcefully  demonstrated  by  time  optimally  regulating  the 
state  vector  of  the  rigid  launch  vehicle  given  In  Flgxore  1. 

When  all  components  of  the  state  vector,  pitch  attitude,  pitch 
rate,  angle  of  attack  and  glmbal  deflection  are  bro^ight  to 
zero  In  minlmiim  time  from  an  initial  displacement  In  pitch 
attitude  of  0.01  radian,  displacements  of  attitude  and  angle 
of  attack  greater  than  0.15  radian  occur.  Although  this  Is  the 

.  *Work  reported  In  Section  I  was  accomplished  under  NASA  Con¬ 
tract  NASr-27. 


tlme-optijnal  response  for  regulation  of  the  state  vector.  It  Is 
certainly  not  acceptable  since  It  would  literally  destroy  the 
vehicle.  On  the  other  hand.  If  the  problem  posed  Is  that  of 
bringing  the  single  component,  pitch  attitude,  to  zero  In  mini¬ 
mum  time  and  holding  It  there  then  the  deadbeat  response  to 
step  Input  of  attitude  Is  obtained  (Plgvire  1 ) ,  In  this  case 
angle  of  attack  and  glmbal  deflection  are  not  zero  at  the  re¬ 
sponse  time  (time  vrtien  6  and  5  are  first  zero)  but  decay  with 
a  21.7  second  time  constant  characteristic  of  the  plant.  It 
has  been  shown  that  single  con^onent  control  can  be  described 
as  motion  to  a  region  In  the  n-dlmenslonal  Space.  The  target 
region  Is  determined  as  that  region  In  n-space  where  the  com¬ 
ponent  being  controlled  Is  zero  and  Is  capable  of  being  held 
there  with  a  bovinded  control  variable.  (Reference  1,  2)  The 
necessary  and  sufficient  conditions  for  mlnlmiun  time  motion  to 
such  a  region  have  been  obtained  (Reference  3). 

In  the  work  presented  In  this  paper,  optimum  control  syn¬ 
thesis  techniques  eu’e  demonstrated  for  control  of  pitch  atti¬ 
tude.  The  controller  obtained  Is  fourth  order,  one  dimensional. 
Hiat  Is,  the  control  variable  Is  a  function  of  foxir  variables, 
and  the  target  set  Is  a  line  segment  in  this  four-space. 

Choice  of  pitch  attitude  was  arbitrary.  The  techniques  apply 
equally  as  well  to  control  of  other  components  of  the  state 
vector  or  to  control  of  a  linear  combination  of  them  such  as 
minimum  drift. 

1.3  A  Truncated  Model  Although  time-optimal  control  theory 
applies  in  principle  to  regulation  of  plants  of  any  order.  It 
is  not  desirable  nor  necessary  to  apply  it  in  controller  design 
to  the  complete  plant  representation  when  the  motion  of  the 
variable  being  controlled  is  primarily  Influenced  by  relatively 
few  variables.  In  the  launch  vehicle  considered,  the  flexure 
mode  f.requencles  are  quite  high  and  aero-dynamic  coupling  small 
so  flexure  has  only  minor  effects  on  rigid  body  pitching  motion. 
The  same  is  true  of  the  actuator  dynamics.  Consequently  there 
is  a  natural  division  of  the  plant  into  a  set  of  dominant  and 
a  set  of  secondary  dynamics.  Time-optimal  synthesis  is  applied 
to  control  the  dominant  modes  only,  and  conceptually  the  second¬ 
ary  dynamics  act  as  a  filter  on  the  primary  modes.  This  is 
shown  in  Figure  2.  The  transfer  function  ©R,  for  the  entire 

u” 

plant  of  Table  1,  has  been  divided  Into  two  parts 


e„(8) 

u(s) 


=  G, 


'1  •  ''2 

Primary  dynamics  are  contained  In 

G,  = 


(  I.  3.  I  ) 


0.8808  (s_+  0.0478; 


'1  ■■  bTs~  'O.lS-^l  ■(T-T.T^ 


s'  +  r.-4-$6^) 


(1.3,  2  ) 


and  secondary  dynamics  In  G^,  Feedback  of  the  fictitious  out¬ 
put  of  Gj^  is  used  for  controiler  design.  The  partial  principle 

coordinate  methods  of  Reference  4  permit  one  to  derive  the 
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linear  transformation  relating  the  y  coordinates  to  the  state 
of  the  system,  x.  The  transformation 

y  =  Lx  ,  (I.  3.3) 

where  y  Is  an  m- vector,  L  an  mxm-matrlx  and  x  an  n- vector.  In 
general,  then  permits  the  fictitious  control  loop  of  Figure  2 
to  be  changed  to  the  one  which  Is  physically  realizable  In 
Figure  3. 


A  plant  In  state  vector  form  which  gives  the  transfer 
function  of  eqviatlon  (1.3.2)  Is, 
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tlons  from  a  set  of  equations  of  this  form  but  with  xmknown 
coefficients  and  then  adjusting  coefficients  to  give  the  pro¬ 
per  poles,  zeros  and  gains.  A  similar  set  of  equations  could 
be  obtained  directly  from  the  transfer  functions  of  equation 
(1.3.2)  and  the  transformation  to  continuous  coordinates  of 
Reference  5. 


The  transformation  matrix  L,  which  relates  the  output  of 
the  flexible  vehicle  to  the  y  variables  contains  many  elements 
which  are  very  small.  It  la  possible  to  neglect  these.  The 
transformation  used  In  the  analog  simulation  was. 
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seen  that  y^^,  corresponds  very  closely  with  0j^,  yg 


R 


R 


a, 


R 


6 

e 

with 


(I.  3.  5) 


with  Opj  and  y^j  with  6^.  Motion  of  9^  corresponds  very 

closely  with  that  of  Sr,,  so  It  Is  reasonable  to  take  equation 
(1.3.4;  as  the  trvincated  model  of  the  full  system. 


Two  points  shovild  be  emphasized  In  the  choosing  of  a  trvin¬ 
cated  model  for  controller  design.  First,  division  of  the 
plant  into  primary  and  secondary  dynamics  cannot  be  made  imtil 
the  variable  to  be  controlled  has  been  specified.  This  vari¬ 
able  may  be  one  of  the  physical  variables  appearing  In  the 
state  vector  x  or  may  be  a  linear  oomblnatlon  of  them.  Second, 
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even  If  the  secondaxy  dynamics  are  a  result  of  a  limited  num¬ 
ber  of  physical  variables  In  the  equations  of  motion  (equation 
1.2.1),  the  primary  dynamics  cannot  be  obtained  by  simply 
neglecting  these  variables  and  equations.  For  example.  If 
the  equations  for  the  flexure  modes  and  actuator  were  omitted 
In  truncating  to  a  fourth  order  model  corresponding  to  0i , 
the  poles  at  -1.4962  and  1.4296  would  be  at  -1.47  and  1.403. 


1.4  Closed-]^op  Time  Optimal  Control  Law  The  next  step  In 
the  synthesis  procedure  Is  to  derive  a  closed-loop  controller 
for  the  model  of  equation  (1.3.4).  The  criterion  for  design 
Is  time-optimal  regulation  of  pitch  attitude;  that  Is  y^  Is 
to  be  brought  to  zero  from  an  Initial  condition  In  minimum 
time  subject  to  a  bovmded  control  variable,  and  then  held  at 
zero.  This  corresponds  to  motion  to  a  one  dimensional  line 
segment  In  the  four  dimensional  space  of  0p,  d^,  a^,  6^, 


There  Is  no  known  method  for  obtaining  a  useful  closed  form 
expression  for  the  closed-loop  control  law  u(x)  which  moves 
the  plant  to  the  desired  line  segment  optimally.  However,  It 
Is  possible  to  compute  open-lOop  solutions  u(t,x(o)  )  for 
any  Initial  condition  x(0)  using  the  computational  techniques 
described  In  Reference  6.  These  techniques  solve  a  set  of 
transcendental  equations  for  a  contixjl  variable  u(t,x(0)  ) 
which  is  constrained  to  satisfy  the  maxlmimi  principle.  Since 
the  maximum  principle  has  been  shown  to  be  a  necessary  and 
sufficient  condition  for  the  optimum  solution,  the  u(t,x(o)  ) 
obtained  is  the  optimal  one.  It  Is  not  practical  to  solve 
the  required  equations  on  line  to  achieve  effective  closed- 
loop  control.  Instead  a  collection  of  open-loop  optimisn 
tx’ajectorles  from  a  set  of  Initial  Conditions  distributed 
evenly  throughout  the  phase  space  region  of  Interest  Is  used 
to  define  a  closed-loop  control  law  by  the  method  described 
In  Reference  7.  Each  of  the  variables  0p,  dp,  ap,  and  6p  Is 
divided  up  Into  32  regions  called  quanta.  A  Boolean  variable 
Xj,  Is  defined  for  each  quantum  (l=  1,  2,  3,  4,  J=l,  2,. ..32). 
The  variable  X4  takes  a  value  one  If  the  measured  magnitude 
of  the  l^^h  variable  Is  within  the  Ji^h  region  and  takes  the 
value  zero  if  the  magnitude  Is  within  any  other  region.  A 
logic  form. 


u(x) 


4 


sign 


32 

Z 

J=1 


(  1.4.  I  ) 


Is  assumed  capable  of  mechanizing  the  control  law  and  the  128 
constants,  XJ,  are  experimentally  adjusted  to  make  u(x)  agree 
with  the  optimum  control  at  discrete  points  on  the  optimum 
trajectory.  This  adjustment  or  training  procedure  Is  shown  In 
Figure  4.  Switch  S  Is  opened  at  t  =  0  and  the  open-loop 
optimal  solution  u(t)  applied  to  the  simulated  plant.  Output 
of  the  plant  x(t).  Is  the  Input  to  the  logical  net  and  the 
output  of  the  net  u(x(t)  ).  Is  compared  with  the  optimum 
control  variable  u(t,x(o)  )  at  discrete  Intervals  of  time. 

If  the  control  variables  are  different  corresponding  to 
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the  Xv  s  which  are  one  for  that  x(t)  are  Incremented  In  the 
direction  to  make  the  sIot  of  their  sum  the  same  as  the  sign 
of  u(t,x(0)  ).  If  u{x(ty  and  u(t,x(0)  )  are  the  same  then  no 
adjustment  is  made.  This  procedure  was  carried  out  on  a 
general  purpose  digital  computer  using  a  set  of  I98  optimum 
trajectories  for  the  plant  of  equation  (1.3.^) j  distributed  in 
the  space, 

0 

-0.12 

-0.1 

-0.12 

Control  variable  comparison  points  were  at  intervals  of  0.1 
second.  As  the  adjustment  is  carried  out.  the  number  of 
differences  (called  errors)  between  u(x(t)  )  and  u(t,x(0)  ) 
is  an  indication  of  the  convergence  of  the  procedure.  The 
per  cent  errors,  100  ot  errors  In  M  Eolnt.s  _  ^3 

as  a  function  of  the  n;imber  of  trajectories  in  Figure  5. 

First  switch  points  are  those  between  t  =  0  and  the  first 
switch  time,  second  switch  points  between  the  first  switch 
time  and  the  second,  etc.  Initial  were  all  taken  to  be 
zero.  It  is  seen  that  errors  drop  very  rapidly  at  first, 
being  less  than  10  per  cent  after  only  100  trajectories.  At 
5000  and  7500  trajectories  the  resolution  of  the  logic  of 
equation  (1.4.1)  is  artiflcally  Increased  by  multiplying  all 
a£'s  by  two.  At  11,000  trajectories  the  s  are  multiplied 
by  a  factor  of  ten.  Typical  closed-loop  control  responses 
using  the  logic  at  the  stages  of  training  shown  in  Figure  5 
are  presented  In  Figure  6.  At  I98  trajectories  the  controller 
has  not  yet  stabilized  the  statically  unstable  vehicle.  At 
2100  trajectories  the  closed  loop  is  apparently  stable  but 
responses  are  poor.  At  11,000  trajectories  responses  closely 
approximate  optimum.  (Limited  hardware  did  not  permit 
evaluation  of  closed-loop  responses  at  13 » 500  trajectories). 
The  logic  of  equation  (1.4.1)  with  constants  at  11,000 
trajectories  is  taken  as  the  closed-loop  controller  for  the 
plant  of  Table  1. 

1.5  Control  of  the  Flexible  Vehicle  A  block  diagram  of  the 
control  system  is  given  in  i'igure  7.  Mechanization  of  the 
logical  net  for  this  optimal  control  of  the  fourth  order 
plant  was  accomplished  using  standard,  commercial  an^ilog  to 
digital  converters  for  quantization  and.  diode-transistor 
logic  in  conjunction  with  standard  ladder  networks  to  form 
the  logic  of  equation  (1.4.1) (Reference  7).  A  linear 
switching  mode  of  the  control  variable  was  used  when  the 
plant  output  was  within  approximately  one  quant  tun  of  the 
target  set.  This  reduced  residual  errors  due  to  switching 
on  a  quantized  switching  surface  and  held  the  plant  within 
the  target  set.  The  linear  switching  used  In  this  mode  was# 
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±  0.12 

<  0.1 

<  0 .12 


(  1 .  4.2) 
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U  =  sign  |ep  +  1.25  0p  +  0.65  ^pj  . 

No  attempt  was  made  to  minimize  the  steady  state  limit  cycle 
with  the  control  variable  in  this  mode. 

Two  schemes  for  measurement  of  the  variables  fed  back  to 
the  controller  were  Investigated.  The  first  measured  the 
state  of  the  system  using  the  method  of  Reference  8  which 
uses  a  complement  of  n  sensors  in  measuring  the  state  of  an 
nth  order  system.  In  the  second,  a  rigid  body  pitch  rate 
signal  was  derived  using  the  phase  blending  technique  of 
Reference  9.  This  provided  a  signal  which  could  be  freed  of 
first  mode  influence,  however,  in  this  case  a  slight  amount 
of  first  mode  feedback  was  Included  in  the  signal  to  damp 
the  first  mode  bending. 

Typical  analog  responses  are  shown  in  Figure  8,  9>  10  and 
11.  Rigid  body  pitch  attitude  responses  are  qqite  similar  for 
rigid  body  feedback  and  for  blender  feedback  of  pitch  rate. 

The  small  amotint  of  first  mode  feedback  (blender  gain  ki=0.9) 
causes  the  first  mode  to  damp  out  with  the  blender  system 
whereas  with  rigid  body  feedback  there  is  a  sustained 
oscillation.  When  the  blender  gain  Ki  was  set  to  cancel  all 
first  mode  feedback  (Ki  =  l.O),  the  blender  system  also 
exhibited  a  sustained  oscillation  of  the  first  mode.  Responses 
to  40-fps  sharp-edged  gusts  are  shown  in  Figure  10.  The 
single  component  attitude  regulator  essentially  Ignores  the 
gust  disturbance  and  maintains  the  desired  attitude.  Figure 
11  Illustrates  response  to  various  command  Inputs.  Although 
the  system  was  designed  to  approximate  time  optimal  regulation, 
it  exhibits  a  very  good  following  capability. 

1.6  Conclusion  It  has  been  shown  that  the  collection  of 
experimental  procedures  and  theoretical  knowledge  is  sufficient 
to  use  a  time -optimal  regulation  criterion  for  rational  design 
of  controllers  for  a  high  order  plant  with  known  coefficients. 
The  synthesis  procedure  includes  obtaining  a  representative 
set  of  open-loop  optimum  trajectories  for  a  truncated  model 
which  is  based  on  the  dominant  dynamics  of  the  plant.  The 
set  of  openvloop  trajectories  is  used  to  define  a  closed- 
loop  control  law  for  the  model*  When  this  controller  is 
applied  to  the  full  plant,  the  output  is  effectively  that 
of  the  optimally  controlled  model  filtered  by  the  secondary 
dynamics  of  the  plant.  The  resulting  controller  is  relatively 
simple.  However  measurement  requirements  are  severe  in  that 
the  entire  state  of  the  system  must  be  measured,  This  is 
feasible  using  the  methods  of  Reference  8  but  undesirable  be¬ 
cause  of  the  large  number  of  sensors  required.  Such  schemes 
as  the  ©rro  blender  give  promise  of  relaxing  these  requirements. 
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Figure  1.  Time  Optimal  Control  of  a  Rigid  Launch  Vehicle 
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Figure  2.  Conceptual  Feedback  for  Controller  Synthesis 


Figure  3.  Actual  Feedback  for  Controller  Mechanization 


Figure  4;  Logic  Adjustment  Procedure 
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FOURTH-ORDER,  ONE-DIMENSIONAL 
TIME-OPTIMAL  CONTROL  OF  THE 


Figure  5,  Fourth-order  Training  Curve 
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Figure  6.  Closed-loop  Responses  at  Three  Stages  of  Training 
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Figure  7.  Block  Diagram  -  Fourth-^ Order  Attitude  Regulation 
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Figure  8.  Attitude  Regulation  -  Response  to  Initial  Displacement  Errors 
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SECTION  II 


THE  EQUIVALENT  MIOTMIZATION  PROBLEM 
A^JD  THE  NEV/TON-RAPHSON:  OPTIMIZATION  METHOD* 

2.1  Introduction  The  purpose  of  this  section  Is  to  develop 

a  second-order  optimization  scheme  for  the  control  optimization 
problem.  The  particular  form  of  the  problem  of  Bolza  which 
is  considered  is  first  stated.  This  is  follovied  by  the  three 
necessary  conditions  for  the  given  form  of  the  problem.  These 
are  used  to  develop  the  reduced  differential  equations  of  the 
extremals,  which  must  be  satisfied  by  the  optimum  path.  It 
is  then  shoim  that  the  solutions  of  these  equations  are 
functions  of  the  independent  variable  and  the  initial  cond¬ 
itions  for  the  problem.  It  is  further  shovm  that  the  solution 
has  continuous  partial  derivatives  of  at  least  second  order  in 
these  variables.  This  immediately  suggests  the  simpler 
equivalent  minimization  problem;  that  of  minimizing  a  function 
of  several  variables  subject  to  constraint  equations  in  those 
variables.  This  problem  has  well  kno\m  necessary  and  suffi¬ 
cient  conditions  for  a  relative  minimum.  These  are  then  used 
to  develop  the  second-order  Newton-Raphson  optimization  method. 
The  sufficiency  condition  for  a  relative  minimum  is  incorpor¬ 
ated  in  this  method.  The  Newton-Raphson  method  for  the  fixed 
end-point  problem  is  then  presented,  followed  by  the  method 
of  steepest  descent.. 

Three  example  problems  are  presented.  The  first  of 
these,  in  v/hich  the  integral  of  the  square  of  the  control 
function  is  minimized,  is  simple  enough  to  allow  an 
analytical  solution.  It  is  shovm  that  a  relative  minimum 
does  exist,  and  furthermore,  that  it  is  a  global  minimum. 

The  presented  concepts  are  also  illustrated  by  this  problem. 

The  second  example  is  the  time  optimal  bang-bang  problem. 

This  is  presented  as  an  example  of  a  problem  with  corners 
v/hich  can  be  treated  by  the  Newton-Raphson  optimizing  scheme. 
The  third  example  is  a  re-entry  problem  which  must  be  solved 
on  a  computer.  The  deviation  of  heating  rate  from  a  given 
constant  is  to  be  minimized.  An  inequality  constraint  on  the 
sensed  acceleration  is  Included  in  the  problem  formulation. 

The  reduced  differential  equations  of  the  extremals  are  de¬ 
rived  for  this  problem. 

A  special  form  for  the  Newton-Raphson  method  is  then 
developed.  This  assiimes  that  the  terminal  surface  is  des¬ 
cribed  by  a  given  terminal  time  alone.  Conclusions  from  the 
application  of  this  method  to  the  third  example  problem  are 
summarized . 

2.2  Problem  Formulation  The  problem  considered  here  is  a  ,  . 
special  form  of  the  problem  of  Bolza  as  formulated  by  Bliss 


*  Work  in  Section  II  was  supported  under  U.  S.  Air  Force 
Contract  AF  33(657)-7383,  ASD. 
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and  extended  to  Include  Inequality  constraints  by  Valentine. 
It  is  desired  to  find  that  path  Which  minimizes  the  function 


(7) 


J  =  e(x(T), 


T  _ 

T)  +  /  f  (x,  u)dT 

^o 


(2.2. 1  ) 


subject  to  differential  equations  of  the  form 

X  =  f(x,u),  x(tQ)  =  Xq  (2.2.2) 

inequality  constraints 

G(x,u)  ^  Z  (2.2.3) 

and  terminal  surface  equations 

f(T,  x(T)  )  =  Z.  (2.2. 4  ) 

In  the  above,  x  and  d  are  n  and  m  dimensional  state  and  control 
co?.umn  vectors  respectively,  and  (•)  represents  differentiation 
with  respect  to  the  independent  variable,  t.  The  dimension  of 
vector  equation  (2.2.3)  is  q,  and  that  of  (2.2.4)  is  r,  v;here 
r  £  n  +  1.  Z  represents  the  zero  vector  of  appropriate 
dimension. 

Following  Valentine,  ^ equations  (2.2.3)  are  rev/ritten 
in  the  form 

=  G(x,u)  (2,2.5  ) 

• 

v;here  the  components  of  ^  are  Oj,  J  =  l,...,q.  The  5' s 
are  real  slack  variable  derivatives,  introduced  to  complete 
the  set  of  differential  equations  for  the  problem  of  Bolza. 

No  more  than  m  of  the  a' s  may  be  zero  at  any  point  on  the  path. 

It  is  assumed  that  fo(x,u)  and  the  differential  equations. 
(2.2.2)  and  (2.2.5)  have  continuous  partial  derivatives  of  at 
least  third  order  in  all  variables  in  an  open  region  Sp  about 
the  minimizing  path.  Furthermore,  the  matrix  made  up  of  the 
partial  derivatives  of  the  differential  equations  with  i-’espect 
to  all  derivatives  and  the  control  functions  must  have  rank 
n+q  at  each  point  of  the  minimizing  path.  This  Insures  that 
the  differential  equations  are  Independent.  The  matrix  has 
the  form 


(2.2.6  ) 


vifhere  I  is  the  hxn  identity  matrix,  Z  is  an  nxq  zero  matrix  Z’ 
is  the  transpose  of  Z,  and  2  is  a  qxq  diagonal  matrix  v;hose 
elements  are  2Sj.  In  addition, 


-  I  '^u^'  2 

Z'  V,G  -2 
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(2.2.7) 
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The  solution,  x(t),  of  the  differential  equations  (2.2*2) 
and  (2.2.5)  is  supposed  to  be  continuous,  v/ith  at  least 
absolutely  continuous  first  derivatives.  In  some  instances 
it  will  be  possible  to  consider  corners,  i.e.,,  points  at  which 
the  derivatives  are  discontinuous.  The  control  functions  are 
treated  as  derivatives  in  (2.2.6)  so  that  they  can  be  dis¬ 
continuous,  as  in  the  bang-bang  problem.  Potential  corners 
are  those  points  at  v/hich  Inequality  constraints  change  from 
greater  than  to  equality  states  or  vice  versa.  They  may  also 
be  defined  by  sv/itching,  points,  as  in  the  bang-bang  problem 
where  the  constraint  is  aTv;ays  zero.  A  subarc  is  defined  as 
that  part  of  the  path  between  corners  or  potential  corners. 


Finally,,  the  functions  g(T,  x(T)  )  and  (2.2.4)  are  assumed 
to  have  continuous  partial  derivatives  of  at  least  third  order 
in  an  open  set  of  points  (T,  x(T)),  and  the  matrix 


di?^ 

d'F^ 

dr’  ’ 

• 

d3c^  4  •••  ^ 

• 

• 

hi 

si 

di’  . 

r  , 

r 

r 

_dF" 

dx]^  >  •  •  •  > 

is  assumed  to  have  rank  r.  An  admissib^e_arc  is  defined  to  be 
a  path,  where  all  of  its  elements  (t,x,x,u)  and  (T,  x(T))  lie 
in  and  respectively. 

It  might  be  remarked  that  differential  equations  and 
inequality  constraints  containing  t  explicitly  can  easily  be 
brought  to  the  form  of  equations  (2.2.2)  and  (2.2,5).  The 
independent  variable  Is  changed  from  t  to  s  by  adding  the 
differential  equation 


ds 


1, 


(2.2.10) 
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(2.2.11  ) 


and  noting  that 

dx  _  d5c  ds  _  die 
3^  ”  3s  316"  ~  3s  * 

Then  t  becomes  a  dependent  variable,  and  the  resulting  n+q-!-l 
differential  equations  are  in  the  desired  form.  It  should 
be  noted  that  the  order  of  the  system  has  been  Increased 
by  one. 

2 .3  The  Necessary  Conditions  and  the  Initial  Value  Problem 
The  multiplier  rule  and  the  necessary  conditions  of  Wlerstrass 
and  Clebsch  are  given  here  for  this  problem  v^lthout  proof. 

The  reader  is  referred  to  references  (l),  (2)  and  (7)  for 
their  derivation.  It  is  then  shovm  that  if  these  necessary 
conditions  provide  unique  descriptions  of  the  control  functions 
over  every  subarc  of  the  path,  then  the  optimization  problem 
is  an  initial  value  problem. 

The  first  necessary  condition  for  this  problem  is 
The  Multiplier  Rule .  An  admissible  arc  E,  defined  on  an 
Interval  'To'^T'  said  to  satisfy  the  multiplier  rule  if 
there  exist^constants  p  =  1,  e*  =  [^-,>...>0  1,  not  all 
zero  and  a  function  ^  J 

•  •  •  • 
F(t,x,u,x,p,ir,a)  =  f^  +  p(f-x)  +  il((?  -o^)  (2.3.1) 

with  multi£llers  p(t)  =  [pi(t) , . . .  ,pj.i(t)l  ,  continuous  on 
rto,Tl  and  |i(t)=  [(^^^(t) ,  I-. .  ,p.q(t)|  contlhuous  on  [tQ-Tj 

except  possibly  at  corners  of  E  v/here  unique  right  and 
left  hand  limits  exist,  and  satisfying  the  Euler- 
Lagrange  equations 


p  =■  -K^'o  + » ■' 

P 

(2.3.2  ) 

z  -  +  P  V  P 

''uO 

(2.  3.  3) 

0  =  J  = 

r  •  •  •  ^  Q 

(  2.  3.  4  ) 

v/here 

i.  >_  fl. 

(2.3.5) 

and  differential  equations 

X  =  f;  xCt^)  = 

■v* 

"  0 

(2.  3.6) 

^  =  G  >  Z 

(2.3.7) 

along  E,, 

and  furthermore,  such 

(  ^0  ^  "  3t  ®  if) 

that  the  equations 

1 

1  dT  + 

(2.  3.8) 
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-I-  (v,^g  -  p) 


dx(T)  =  0,  1  Z 


hold  for  everj’-  choice  of  the  differentials  dT,  d>:(T).  The 
multipliers  Pq,  p,  ir  do  not  vanish  simultaneously  at  any 
Dolnt  of  the  interval  [tg  t]  for  an  arc  E  satisfying  the 
multiplier  rule.  Furthermore,  the  function 


H  =  f^  -t-  pf 
o  ^ 


(2.3.9) 


is  a  constant  on  E.  Every  minimizing  arc  E^  for  the  given 
form  of  the  problem  of  Bolza  must  satisfy  the  multiplier 
rule . 

In  the  above ^ 

Sf_  Bf. 


V  f  = 

X  0 


V,,g  = 


V.J  = 


A’.r  >  •  •  •  ) 


n 


Sg 

_  ^  ‘^^n^ 

'df^  df^' 

3x7  ’  33^ 

•  -L  •  n 


df  df 

n  n 

33r 

n 


337 


V  f  = 
u  o 


df  df 

0  o 

337  '  *  ^337 

1  m 


dG^ 

3337 

•  -L 

si 

33cf 


dG, 


3X 

n  • 

aa 

» •  •  • 


(2.3.10) 


(2.3.11) 


with  V^f  and  V^^'5’  defined  by  ^equations  (2.2.7)  and  (2.2.3) 
respectively.  The  vector  is  the  first  column  of  the 

matrix  (2, .,2.9)  ^ind  is  the  remainder  of  the  matrix. 

The  vanishing  of  the  coefficients  of  dT  and  d3c(T)  in 
equation  (2.3.8)  constitutes  the  transversality  condition  on 
the  arc  E.  Thus, 


H-  -  (fo  +  SDt  =  -  (f  +  e  ||) 


T 


p(T)  =  (v^g  +  eVl) 


T* 


(2.3.12) 

(2.3. 13) 


The  Hamiltonian,  equation  (2.3.12),  is  often  called  a  first 
Integral  for  this  problem.  It  has  further  significance  in  the 
second  necessary  condition. 


The  Necessary  Condition  of  Welerstrass  (The  Minimum 
Principle),.  An  admissible  arc  E  s'atisfyin/^  the  multiplier 
rule  with  multipliers  Po  =  1,  P,  \i.,  is  said  to  satisfy 
the  necessary  condition  of  Welerstrass  with  these 
multipliers  if  the  condition 
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(  2.  3.14  ) 


\ 


H(t,x,p,u)  <  H(t,x,p,TJ) 

• 

is  valid  at  everjr  e^l.£ment  (t^Xj_x_^u)  of  E  for  all  admis¬ 
sible  points  (t,x,X,U)  ^  (t,x,x,u)  satisfyinc  the  equations 
(2.2.2)  and  (2.2.3).  Every  minimizing  arc  E  for  the  given 
form  of  the  problem  of  Bolza  must  satisfy  this  condition. 

A  consequence  of  the  V/eierstrass  condition  is 

The  Necessary  Condition  of  Clebsch.  At  each  point  of  E 
letjCr  be  a  vector  v;hose  components  are  those  components 
of  G  which  vanish,  and  'ii  the  corresponding  multipliers. 

Then  the  inequality 

W  (H  +  113^  ~  ^  0  (  2.  3.  I  5  ) 

must  be  satisfied  for  every  vector  tF  ^  Z  v/here 
tt'  =  satisfies  the  equations 

V^Gw  =  Z.  (2.3.16) 

Every  minimizing  arc  for  the  given  form  of  the  problem  of 
Bolza  must  satisfy  this  condition. 

In  equation  (2.3.15)  the  operator  is  the  matrix 


u 


>  •  •  •  9 

au?  au, au 

•  J.  JL  1 


a^ 

3uJ^ 


1 


P  •  } 


It  v/lll  now  be  shovm  that  the  proble.m,  can  be  considered 
to  be  an  initial  value-  problem.  For  this  purpose,  the  arc  is 
split  up  into  subarcs.  On  each  subarc  a  given  subset  of  the 
inequality  constraints  are  equality  constraints,  and  all  the 
rest  are  greater  than  zero,  except  possibly  at  a  finite 
niimber  of  points,  It  is  supposed  that  there  are  no  corner 
points  anywhere  on  the  path.  For  simplicity,  it  will  be 
assvimed  that  all  the  constraints  are  greater  than,  zero  along 
the  first  subarc.  Then  according  to  Valentine,'' '  the  in¬ 
equality  constraints  may  be  neglected  over  this  subarc.  The 
function  (2.3.1)  fiiay  thus  be  v/rltten 

?!  =  fo  +  P(^  -  x)  (2.3.17) 

The  corresponding  Euler- Lagrange  equations  (2,3.2)  and  (2.3.3) 
are 

P  -  +  P  1  '  Vl  <  2  3.18) 

^  =  Vo  +  5  V  "  Vi- 
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(2.3.  19) 


It  Is  v;ell  loiovm  from  irapllcit  function  theorv  that  equations 
(2.3.19)  can  be  solved  (at  least  in  principle)  in  the  foria 


u  =  u(3c,p) 
if  the  determinant 


=  det  Vj  =  det 


3=P, 


}  •  •  •  y 


c)Ut  du 
.1  m 


d^F, 


du^du, 
m  1 


9  •••  y 


(2.3.  20) 


(2.  3.  2l  ) 


is  different  from  zero.  Equation  (2.3.21)  is  the  determinant 
of  the  Hilbert  differentiability  condition  for  this  problem. 
Substitution  of  (2.3.20)  into  (2, 3.13)  and  (2.2.2)  thus 
produces  a  2n  set  of  differential  equations 
•  ■  _ 

X  =  fj^(x,  p),  x(t^)  =  x^  (2.3.22) 

F  =  P)  (2.3.  23  ) 


These  will  be  called  the  reduced  differential  equations  of  the 
extremals .  It  is  well  Imovm  from  the  theory  of  differential 
equations  that  (2.3.22)  and  (2.3.23)  have  a  unique  solultlon  for 
a  given  set  of  initial  conditions.  But  half  of  these,  x  ,  arc 
given  for  the  problem. _  The  solution  is  thus  a  function  8f  the 
n  initial  conditions,  Pg,  and  is  said  to  be  imbedded:  in  an  n- 
parameter  family  of  extremals.  Over  the  first  subarc,  then  the 
solution  has  the  form 

X  =  x(t,XQ,PQ)  =  x(t,pQ)  (2.3.24) 

p  =  p(t,XQ,PQ)  =  p(t,pQ)  (2.3.25) 

u  =  u(t,XQ,PQ)  =  u(t,pQ)  (2.3.  26) 

It  is  further  knovm  from  the  theory  of  differential  equations 
that  equations  (2.3.24)  and  (2.3.25),  and  consequently  (2.3.26), 
have_contlnuous  partial  derivatives  in  the  variables,  t,  x 
and  po  of  at  least  second  order. 

Before  continuing,  it  should  be  remarked  that  the  above 
results  are  of  course  obtainable  from  the  general  form  of  the 
problem  of  Bolza.  It  is  knov/n  that  the  arc  can  be  Imbedded 
in  a  2n  +  2m  parameter  family  of  arcs,  and  that  there  are 
2n  +  2m  differential  equations  of  the  extremals.  The  2ra 
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differential  equations  become  the  m  algebraic  equations 
(2.3.19)  and  their  time  derivatives  (which  introduce  no 
nev;  information)  v/hen  initial  conditions  are  Imposed.  Further¬ 
more,  the  determinant  in  the  Hilbert  differentiability 
condition  reduces  to  the  form  (2.3.21). 

Over  the  second  subarc  it  is  assumed  that  one  of  the  in¬ 
equality  constraints,  say  G]_,  is  equal  to  zero.  The  function 
(2.3.1)  is  written,  following  Valentine , v7 )  as 

Pg  =  ^o  P  ^^1®1  (2.3.27) 

and  the  Euler-Lagrange  equations  as 

p  =  -  [  +  P  -I-  Gj.]  =  (2.  3,28  ) 

^  P  V  +  Pi  ’u°i  -  Va  '2-  3. 29 ) 


To  this  is  added  the  equality  constraint 


0 


(2.3.30  ) 


If  the  m-f-1  equations  (2_.3.29)  and  (2.3.30)  are  to  be  solved 
for  the  m+1  variables  u  and  the  determinant 


Rg  =  det 


V^F 
u  2 


V  G, 
u  1 


V'G, 

u  1 


(  2.  3.  31  ) 


must  not  be  zero.  Here,  V'  is  the  transpose  of  the  operator 
Vu.  Equation  (2.3.31)  is  ^  the  determinant  of  the  Hilbert 
differentiability  condition  for  this  arc.  It  is  then  found 
that 


u  =  u(x,  p) 


(2.3. 32  ) 


P) 

and  consequently  that 

« 

X  =  72(x,  p) 

P  =  P)’ 


(2.  3.  33  ) 


(2.  3.  34) 


(2.  3.  35  ) 
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the  reduced  differential  equations  of  the  extremals  for  this 
arc.  These  equations  again  have  a  unique  solution  for  a 
given  set  of  initial  conditions.  Furthermore,  the  solutions 
possess  continuous  partial  derivatives  of  at  least  ^econd  _ 
order  with  respect  to  t  and  the  initial  conditions  x-]_  and  P]_ . 
It  remains  to  be  shovm  that  the  solutions  over  this  arc 
are  continuous  functions  of  the  initial  conditions,  Pq,  and 
that  partial  derivatives  of  at  least  second  order  exist. 

The  terminal  point  of  the  first  subarc  is  defined  by  the 
equation 


G^(x  (t^,p^),  p(t^,p^))  =  0.  (2.3.36) 

This  may  be  solved  for  t^(p^)  if  the  derivative 


°1  = 


^x^l 


X  -h 


'pi 


p' 


is  non-zero.  Equation  (2.3.36)  represents  the  equation  for 
the  terminal  surface  of  the  family  of  extremals  which  are 
solutions  of  equations  (2.3.22)  and  (2.3.23).  This  terminal 
surface  represents  the  initial  surface  for  the,  family  of 
extre.mals  over  the  second  subarc  which  are  solutions  of 
equations  (2.3.3^)  and_(2 . 3^3.3) .  _For  a  given  set  po,  the 
terminal  values  x(ti,  Po),  p(ti,  Po)  ^^re  the  irtb.lal  values 
for  the  differential  equations  _(_2.3.3‘^)  and  (2.3.35).  This 
follows  from  the  continuity  of  x(t)  and  p(t).  Then  since 
the  solutions 


x(t)  =  x(t,x^,p^),  t  >_  t^  (2.3.37) 

p(t)  =  p(t,x^,p^),,  t  ^.t;^  (2.3.38) 

are  continuous  functions  of  and  p^,  and  since 

Xi  =  x(t^,  Pq)  =  x^(Po)  (2.3.39) 

Pi  =  P(ti,  Pq)  =  Pi(Pq)  (2.3.  40) 

are  continuous  functions  of  p^  alone,  it  follov/s  that 

X(t)  =  x(t,  p"^),  t  >_  t^  (2.3.  41  ) 

p(t)  =  p(t,  Pq),  t  :^t^.  (2.3.42) 


Furthermore,,  since  continuous  partial  derivatives  of  at  least 
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second  order  exist  for  equations  (2,3,37)  -  (2. 3.^0)  in  the 
Indicated  variables,  it  follov/s  that  (2,3.41)  and  (2.3.42)  _ 
possess  partial  derivatives  of  at  least  second  order  in  t,  Pq. 
Since  there  are  no  corners  j  jx,  is  at  least  absolutely 
continuous . 

Further  siibarcs  nay  novj  be  added,  so  Ions  the  number 
of  equality  constraints  does  not  exceed  m,  the  dimension  of 
the  control  vector.  Each,  subarc  possesses  control  functions 
and  multipliers  of  the  form  (2.3.32)  and  (2.3.33)  j  £iad 
furthermore,  differential  equations  similar  to  (2.3.34)  and 
(2.3.35).  It  is  readily  verified  that  the  solutjLons  are 
continuous  functions  of  the  initial  conditions,  Pg,  and 
that  continuous  partial  derivatives  of  at  least  second  order 
exist . 

Points  where  equality  constraints  change  to  the  "greater- 
than"  state  will  now  be  examined.  For  this  purpose  it  v/lll  be 
assumed  that  only  one  of  the  constraints,  saj/  Gt  ,  is  zero 
over  the  first  subarc,  and  that  it  is  greater  than  zero  over 
the  second.  Equations  (2.3.32)  -  (2.3.35)  hold  over  the  first 
subarc,  and  (2.3.20),  (2.3.22)  and  (2.3.23)  over  the  second 
subarc.  The  terminal  surface,  (2.3.3^),  is  replaced  by  the 
equation 

M.^(x  (t^,p^),  p(t^,p^))  =  0  (2.3.43) 

since  m-j  is  a  continuous  function,  aiid  must  go  to  zero  before 
Gq  can  be  greater  than  zero.  It  is  then  seen  that  the 
arguments  follow  through  as  before,  provided  the  doter.minants 
Rq  and  Rp  are  different  from  zero,  and  that  the  derivative 

»■  .*  • 

is  non-zero  on  the  terminal  surface  of  the  first  subarc.  On 
the  second^  subarc,  then,  the  solution  is  a  continuous  function 
of  t  and  Pq,  and  partial  derivatives  in  these  variables  of 
at  least  second  order  exist.  The  res'u.lt  is  easily  generalized 
to  several  equality  constraints  going  to  the  greater  than 
state  over  a  series  of  subaresa 

To  sum  up,  a  path  is  composed  of  a  finite  number  of  sub¬ 
arcs,  each  of  which  has  its  own  set  of  differential  equations 
and  terminal  surfaces.  The  first  subarc  has  a  specified  set 
of  initial  conditions,  Xq^  and  the  last  one  has  a  specified 
terminal  surface,  equations  (2.2.4).  The  ecraations  of  this 
terminal  surface  heive  continuous  partial  derivatives  of  at 
laast  second  order  v;lth  respect  to  the  initial  conditions:, 

Po,  and  the  terminal  value  of  the  Independent  variable,.  T. 

2 .4  The  Equivalent  Hinimizatlon  Problem  It  will  be  shovm 
here  that  the  minimization  problem  can  be  redi;cod  to  the 
more  familiar  one  of  minimizing  a  function  of  sevex’al 
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variables  subject  to  constraint  equations  in  those  variables. 
For  this  purpose,  the  integral  term  of  equation  (2.2,1)  is 
vjritten  as  a  differential  equation 


^n+1  ^o’ 


"n+1 


(to)  =  0 


(2.4.1  ) 


and  added  to  the  differential  equations  of  the  extremals. 
The  solution  to  this  is  easily  seen  to  be  of  the  form 


n-l-l 


11+1 


(t,p^)  = 


t 

I  fo(^,p)dT 

t„ 


(2.4.2  ) 


over  the  path,  and  on  the  terminal  surface, 

'Si+l(^)  =  ^'n+l(^>Po)-  <2.4.3) 

Furthermore,  since  g  of  equation  (2.2.1)  is  a  functional  of 
continuous  functions, 

J(T)  =  J(T,p^)  =  g(T,  I(T,p^))  +  (T.Fq)  (2.4.4) 


The  terminal  surface  equations  (2.2.4)  may  be  expressed  in 
the  form 

f(T)  =  Y(T,Pq)  =  ^(T,  x(T,Fo))  =  Z.  (2.4.5) 

It  is  assumed  for  the  present  that  the  dimension,  r,  of 

(2.4.5)  is  ^  n.  The  minimizing  arc  must  satisfy  the  three 

necessary  conditions,  and  hence  the  differential  equations 

of  the  extremals.  S^nce  the  solutions  are  functions  of  the 

initial  conditions,  Pq  and  t,  the  problem  is  thus  one  of 

minimizing  (2.4.4)  in  the  variables  T,  p  ,  subject  to  the 

constraint  equations  (2.4,5).  This  is  a  problem  for  v/hich 

necessary  and  sufficient  conditions  for  a  relative  rulnlmum  ; 

are  v/ell  knovm.*  The  first  necessary  condition  requires  1 

that  the  partial  derivatives  of  the  function  j 

j 

P  =  J(T,  Fq)  +  4  ^  (T,  Fq)  (2.4.6)  i 

with  respect  to  T,  Fn>  sill  vanish  at  the  minimizing  point. 

Here  =  , ...,Xr|  is  a  set  of  constant  undetermined  1 

Lagrange  multipliers.  One  thus  finds  i 


*  for  example,  see  Bliss,  reference  (2),  pp.  210-211. 
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1 


(2.4.7  ) 


If  ”  +  *3f  *  ''xS  *)  +  ^  (H  ’x  “  ° 

■^o  o  •  o  o 


v/here  V  x  is  the  matrix  composed  of  partial  derivatives  of 
^o 

the  functions 


xx(t)  =  x(T,  p^)  (2.4.9) 

v;ith  respect  to  the  initial  conditions  p^.  This  nxn  matrix 
is  non-singular. 

Equations  (2.4.7)  and  (2.4.3)  must,  of  course,  be  re¬ 
lated  to  the  boundary  condition  (2.3.3)  of  the  multiplier 
rule.  The  relationship  can  be  found  by  substituting 


dx(T)  =  X  dT  +  V  X  dp  (2.4..  10) 

Po  ° 

into  the  boundary  condition  (2.3.3)  and  rearranging.  Here, 
dp^  is  a  colvimn  vector.  This  gives 


Vanishing  of  the  coefficients  of  dT  and  dp^  in  this  equation 
is  equivalent  to  the  transversality  condition.  Comparison  of 
(2.4.7)  v/ith  the  coefficient  of  dT  in  (2.4.11)  then  shov/s 
that 


A  = 


(2.4.  12) 


i.e.,  the  undetermined  Lagrange  .multipliers  of  the  boundary 
condition  for  the  problem  of  Bolza  are  the  ca.me  as  those  for 
the  equivalent  problem.  Furthernore,  from  (2.4.3)  and  the 
coefficients  of  dpo  of  (2.4.11),  one  finds. 


7  X  ,,  ==  -n(T)  V,^ 
Po  n-:-l 


(  2:4.  13) 
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If  X  , ,  is  assumed  to  be  a  function  of  the  state  of  the 
n+x 

system,  i.e.. 


X 


n+1 


=  x(T)), 


(2.4.14) 


then  equation  (2.4.13)  can  be  rev/ritten  in  the  form 


\  -‘n+l  V  V 

o  o 

Since  V  x  is  non- singular,  it  is  seen  that 
^o 

\  =  -p(T) .  (2.4.  16) 

Equation  (2.4.16)  is  a  v;ell-lcno\vn  result v;hich  finds  use 
in  dynamic  programming.  It  is  further  knovm  that  (2.4.16) 
is  valid  for  all  values  of  the  independent  variable,  t.  Thus, 
equation  (2.4.13)  reduces  to  an  identity. 

Equations  (2.4.6)  -  (2.4.0)  v;lll  nov/  be  v/ritten  in 
simplified  form.  Using  equation  (2.4.12)  and  the  chain  rule 
for  partial  derivatives,  it  is  found  that 


F  =  J(T,  Pq)  +  e  T(T,  pJ 


ap 


=  f^  +  g  +  e  f  =  0 


VF=V  X.,  +V  g+iv  H 

Pq  n+1  Pq^  Pq 


=  z 


(  2.4.  17  ) 
(2.4.  18) 
(2.4.  19) 


It  will  further  be  assumed  that  the  problem  is  normal,  i.e., 
that  the  matrix 


1' 


r' 


<5Po 
■Oi 


f  •  •  •  » 


'n 


,  ...  , 


'n 


12.4.  20  ) 
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is  non-singular.  This  is  equivalent  to  stating  that  the 
solution  Is  normal  for  the  problem  of  Bolza.  Then  the 
sufficiency  condition  for  a  relative  minimum  for  the  equi¬ 
valent  problem  is  that  the  quadratic  form 


- - 1 

< 

o 

_ 1 

- 1 

J 

o 

V  F  F 

n 

o 

o 

■-  J 

».  — 

be  greater  than  zero  for  all  non-zero  vectors 
=  ^l’***'^nj  satisfy  the  equations 


( 2 .4,  2  I  ) 


n 


z. 


In  the  above. 


(2.4.  22  ) 


F  =  f^  +  g  +  e  7 


P  =  V  (f  +  g  e  7). 


This  form  of  the  sufficiency  condition  is  rather  awkward 
to  satisfy.  An  equivalent  condition,  v.'hlch  reduces  to 
shov;lng  that  a  matrix  is  positive-definite,  will  now  be 
developed. 

The  set  (2.4.5)  is  extended  to  nn-l  equations  by  adding 
equations  of  the  form 


p^) 


J=r'hl,  .  .  .  ,n-!-l  (2.4.23) 


where  b  is  a  parameter  which  is  zero  at  the  minimizing  point, 
(b,  po .  These  equations  are  to  be  chosen  such  that  the 
(n-i-l)x(n+l)  matrix 


r-l-1 


^n+l 


V 


rM-l 


1 


n-;-l 


(2.  4.24) 
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Is  non-singular.  This  is  equivalent  to  completing  the  set 
(2.2.4)  vfith  the  equations 


^rj(T,x(T))  =  ITjCT,  x(T))  +  bCj,  J=r4-1, . . .  ,n-l-l  (2.4.25) 
since  the  transformation 


dT 

dx(T) 


1 

x(T) 


Z 

V 


x(T) 


’dT  1 

1 

o'P' 

1 _ 

(2.4.26) 


must  be  non-singular.  Bliss 


(2) 


shovis;  that 


4  • 


(2.4.27  ) 
(2.4.  28  ) 


Then  differentiation  of  equations  (2.4.5)  and  (2.4.23)  v/lth 
respect  to  the  parameter  b  gives 


(2.  4.  29) 


v/here  Z  is  the  rxl  zero  vector  and  ^  is  an  (n+l-r)xl  column 
vector  whose  components  are  Cj.  is  the  matrix  (2.4.24). 
Then  since  this  is  non-singular,  (2.4.29)  niay  be  rewritten 


(  2.  4.30  ) 


Equation  (2.4.30)  may  be  thought  of  as  a  transformation  v/hlch 
relates  the  tj's  to  arbitrary  q.  Substitution  of  (2.4.30) 
into  (2.4.21)  then  gives 


(c'b 


t  v^P 


0. 


(2.4. 31 ) 


Call  the  product  matrix  of  (2.4.31)  A,  and  partition  it  Into 
four  sub-ma,trlces,  A^-A^^.  Then  (2.4.31)  becomes 


Ai  Ag 

'Z  ■ 

.A3 

-T- 

(  2.  4.  32  ) 
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Upon  performing  the  matrix  multiplication,  it  is  found  that 
(2.4.32)  reduces  to 


T  >  0  (2.4.  33  ) 

Since  If  is  arbitrary,  equation  (2.4.33)  is  true  if  and  only 
if  the  matrix  A4  is  positive-definite.  This  is  a  very  useful 
form  of  the  sufficiency  condition  for  the  equivalent  mini¬ 
mization  problem. 

2.5  The  Newton-Raphson  Scheme  A  second  order  scheme,  based 
upon  the  equivalent  minimization  problem,  is  presented  here. 

It  is  suitable  for  computer  solution  of  the  optimization  prob¬ 
lem,  and  has  the  advantage  that  the  optimization  process  can 
be  made  to  be  automatic.  The  reason  for  this  is  that  both  the 
direction  and  the  amount  by  vjhlch  to  move  the  solution  can  be 
computed.  Gradient  schemes,  on  the  other  hand,  provide  only 
the  direction  of  changes.  Human  intervention  is  generally  re¬ 
quired  to  assertain  the  cimount  of  change  from  step  to  step. 

At  this  point,  the_subscript  on  p^  v;ill  be  dropped,  since 
only  initial  values  of  p  will  henceforth  be  considered.  It 
v;ill  be  assumed  that  a  non-optimal  solution  for  the  reduced 
differential  equations  of  the  extremals  has  been  found  for  the 
initial  values  p.  One  of  the  components  of  7,  say  ,  has 
been  used  as  the  stopping  condition  on  this  path.  Thus,  the 
condition 


=  0  .  (2.  5.  I  ) 

is  satisfied,  and  furthermore,  the  terminal  value  of  the 
independent  parameter,  T,  is  Icnovm.  The  other  components  of 
T  are  not  necessarily  zero,  but  their  values  are  laiov/n.  It 
is  further  assumed  that  the  partial  derivatives  in  (2.4.20} 
and  (2.4.21)  are  loiown.  A  method  for  estimating  these  for 
the  general  problem  is  given  belov/.  It  is  desired  to  compute 
new  initial  values,  f),  v;hich  v/ill  simultaneously  satisfy  the 
terrainal  conditions  (2.4.5)  and  minimize  the  function  J(t) 
of  equation  (2.4.4).  The  terminal  value  of  the  Independent 
variable  for  this  path  will  be  designated  T..  Taylor's  series 
expansion  of  the  functions  J  and  T  through  second  order  terms 
then  gives  the  approximate  equations 

J(T,p)  =  J(T,p)-l-J(T,F)(T-T)+Vpj(T,F)(p-p) 

-I-  (T-T)Vpj(T,p)(p-p)4  J(T,P)(T-T)^ 

+  I  (P-P)  J(T,P)  (P-P)  (  2:.5.,2  ) 
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'r^(T,p)  0  =  7^(T,p)+'F^(T,,p)(T-T)+Vp'?^{T,p)(p-p) 


-1:  ( T-T  )  Vp ( T ,  p )  ( p -p )  4^^ ^  ( T ,  p )  (T-T )  ^ 
+|-(p-p)''7p'?i(T,p)(p-p),  i=l,....,r.  (2.5.3) 

A  _ 

Here,  p,  p  are  column  vectors.  These  quadratic  equations 
approximate  conditions  at  the  minimizing  point.  One  can  thus 
write  the  function  F  of  (2.4.6)  and  apply  the  necessary  and 
sufficient  conditions  of  the  equivalent  problem.  Thus 

P  =  J(T,i3)  +  e  T  (T,  p).  (2,5.4) 


It  is  convenient,  at  this  point,  to  introduce  the  function 


-  J(T,p)  +  e  f  (T,p). 


(2. 5.5) 


Pi  has  partial  derivatives  made  up  of  the  partial  derivatives 
in  equations  (2.5.2)  and  (2.5.3).  Thus, 

=  i(T,p)  +  e  t  (T,p)  (2.5.6) 

Vl  "  (2.5.7) 

and  so  forth.  Then  the  first  necessary  condition  for  equation 
(2.5.4)  gives  the  equations 

0  =  Pi  +  Pj^(T-T)  +  VpF^(p-p)  (2.5.8) 

Z  =  VpP^+VpP^(T-T)  +  VjF;^(p-p)  (2.5.9) 

These  equations,  together  v/ith  (2 .5.3)  constitute  n+r-H 
equations  in  the  n+r+1  unlcnovms,  p,  W,  T.  The  interated 
solution  of  these  equations  then  constitutes  the  Newton- 
Raphson  method,  provided  that,  for  the  matrix 

V2p  =  V^F^  (2.5.10) 

the  matrix  A4  of  (2.4.33)  is  positive-definite.  Otherwise, 
the  solution  would  be  driven  tov/ard  a  saddle  point,  or  worse. 


dp 

St 

V'  P  = 
p 
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a  maxlmlzlnB  point. 

If  the  dimension  of  T  is  n+1,  the  method  reduces  to  the 
sin^ler  and. more  familiar  form  of  the  Nevrton-Raphson  method. 

In  this  case,  one  is  trying  to  determine  the  minimizing  path 
which  passes  through  given  Initial  and  terminal  points. 
Satisfaction  of  the  three  necessary  conditions  for  the  problem 
of  Bolza,  and  hence,  the  reduced  differential  equations  of 
the  extremals,  insures  that  the  path  will  indeed  be  the 
minimizing  path.  The  problem  then  reduces  to  one  of  finding 
those  initial  values,  p,  vjhlch  satisfy  the  n+1  terminal  con¬ 
ditions  (2.4.5).,^  The  Nev/ton-Raphson  equations,  to  be  solved 
iteratively  for  p,  T,  are  then 

=  0  =  ^^(T,?)  +  'F^(T,F)('r-T) 

+  Vp'i'^(T,p)(p-p),  1  =  1,..  .,n+l  (2.5.11) 

Truncation  of  these  equations  after  first-order  terms  in  the 
expansion  is  a  consequence  of  seeking  a  root,  rather  than  a 
minimizing  point.  The  matrix  (2.4.20)  must  be  non-singular 
for  the  method  to  work  at  all. 

It  might  be  remarked  that  some  pre-dlgesting  of  the 
problem  can,  be  done  to  eliminate  the  variable  T.  If  equation 
(2.5.1)  is  the  stopping  condition  on  the  problem,  then  the 
equation  (2.5.3)  for  can  be  used  to  perform  the  elimination. 
The  result  is  an  n+r^ system  of  equations,  (2.5.8)  and  (2.5.9), 
in  the  n+r  unloiovms  p,  e.  The  elimination  is  more  readily 
performed  for  the  fixed  terminal  point  problem,  and  the  method 
of  steepest  descent  which  follov/s. 

2.6  The  Method  of  Steepest  Descent  If  the  original  guess  on 
the  initial  conditions,  p7  is  too  far  from  the  minimizing  „ 
values,  p,  equations  (2.5.2)  and  (2.5.3)  will  not  apply.  In 
this  case,  the  matrix  A4  of  equation  (2.4.33)  will  not  be 
positive-definite.  The  simpler  method  of  steepest  descent 
may  then  be  used  to  drive  tov/ard  v;hat  is  hopefully  the 
minimizing  point.  Equations  (2.5.2)  and  (2.5.3)  are  truncated 
after  first  order  terms,  and  v/ritten  in  the  form 


dJ  =  j(T,p)  dT  +  Vpj(T,p)  dp  (2.6.  I  ) 

dT  =  t(T,p)  dT  +  VpT(T,p)  dp  (2.6.2) 

dT  =  T^T  (2.  6.3) 

dp  =  p  -  p  (2.6.4  ) 

dJ  =  J(T,p)  -  J(T,p)  (2.6.  51 

dT  =  T(T,p)  -  f(T,,F)  (2.6.6) 
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v/here 


and  T,  p  are  better  estimates  of  the  minimizing,  point  v;hlch  is 
to  be  determined.  Equations  (2,6.1)  and  (2.6.2)  may  be  written 
as  a  vector  equation 

d^  =  Vi?"  dy  (2.6.7  ) 

where 

d9  =  dJ  dy  =  dT]  = 

_dT  dp 

then  the  change,  dy,  is  chosen  in  the  gradient  direction  as 


(2.6.9)  dy  =V'^K  (2.6.9) 

vfhere  is  the  transpose  of  V^,  and  K  is  an  (r-il)xl  constant 
vector  to  be  determined.  Substitution  of  dy  from  (2.6  .,9  into 
(2.6.7)  then  gives 

(2.6.10)  d?"  =  V'^  K.  (2.6.10) 

The  (r+l)x(rH-l)  symmetrical  matrix,  V9V'0,  is  in  general  non- 
slngulai'*,  but  is  singular  at  the  minimizing  point,  p,*?.  Then 
excluding  singular  points, 

IC  =  (V^V^)"^  (2.6.  M  ) 

The  required  change,  from  {2.6.9},  is  thus 

dy  =  V'^(V^  d^,  (2.6.12) 


In  using  this  method,  appropriate  changes,  d^,  are  selected. 
This  is  genei’ally  done  manually.  Then  the  new  estl.mates  of  the 
initial  values,  %,  are  computed  from  equations  (2.6.12),  the 
second  of  (2.6.8j  and  (2.6.4).  It  should  be  noted,  however, 
that  the  method  blov/s  up  as  the  minimum  point  is  approached. 

2,7  The  Partial  Derivatives  For  only  the  simplest  of  problems 
is  it  possible  to  oo.mpute  all  of  the  required  partial  derlYa-.. 
J^lves  g^xpllcitly.  Those  which  can  alv/ays  be  computed  are  J,  j, 
'H  and  y,  since  analytical  expressions  for  these  are  available. 
The  other  partlals  v;ill,  in  general,  have  to  be  approximated. 
There  may  be  several  vrays  of  doing  this.  The  method  given  here 
is  suitable  for  computer  solution,  and  corresponds  to  the  re¬ 
sults  of  differentiating  a  Lagrange  three  point  interpolation 
formula.  .  . *  ' 


It  is  assumed  that  a  solution  of  the  reduced  differential 
equations  of  the  extremals  for  the  initial  conditions,  p,  is 
loiov/n,  which  satisfies,  the  stopping  condition,  equation  (2.,5.l). 
This  establishes  a  terminal  yalue  of  the  independent  variable, 

T,  as  well  a£  values  for  J,  j,  i  and  i.  These  will  be  denoted 
To^  Jq,  jg,  Tq  and  respectively.  Now  suppose  that  a  new 

solution  of  the  reduced  differential  equation  of  the  extremals 
is  found  for  the  initlai  conditions  J^p^^  +  6pj^,  ..., 

vfhere  6pj  Is  a  small  positive  increment.  The  stopping  condi¬ 
tion  for'^this  path  is 
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t  =  T. 


(2.7.1  ) 


gind  the  value  of  J,  J,  'H  and ^  vvlll  be  denoted  J(p,  +  bp.,), 
J(p^+6p,),  ^('(p,  +  6p  )  and  '?(p,  +  6p,).  It  is  noted  that  of 
the'^n+l  variables  T,  p,  only  p,‘*'has  been  changed  to  obtain 
these  values.  A  third  solution  may  now  be  obtained  for  the 


P2> 


and  stopping  condl- 


initial  conditions  -  bp^>  p2J  Pj^j  3.nd  stopping  condi¬ 

tion  (2.7.l)._The  values  for^thls  path  are  denoted  J(Pj  -  Bpj), 
J(P]L  -  SP2_)>  'i''(P^  -  Sp^)  and  ’i'(p^  -  6p^).  These  vaiues'^wlll  be 

used  to  determine  the  partial  derivatives. 

Now  consider  the  function  J.  It  may  be  expanded  in  a 
Taylor’s  series  through  second  order  terms  to  give 


Similarly, 


J(Pl  +  Sp^)  =  Jq 


J{Pl  -  5Pi)  =  Jq  " 


1  ^ 
^  ^Pf 


1  ^ 

2  .a 


(6Pl)' 


(6Pl)^ 


(2.7.  2) 


(2.7.3  ) 


Solution  of  these  equations  gives 

dJ  _  J(Pi+^Pl)  -  J(Pi-5Pl) 


(2.7.4  ] 


52j  _  J(p^+6pj^)  +  J(p^-&p^)  -  2J^ 

ap2  (bPl)^ 

These  equations  are  generalized  to 

aj  _  J(Pi+6Pi)  -  J(Pi-6p^)  ^ 


J^Pi-^-^Pj)  J(Pi-^Pi)  -  gJp  , 
ap|  (Bp^)^ 


1=1, . . . ,n 


(2.7.5) 


(  2.  7.6) 


(  2.7  7) 


by  perturbing  the;  ith  element  of  p  by^  the  amount  -  6p. .  Simi¬ 
lar  equations  can  be' written  for  J,  1'  and  1'.  Equations  (2.7.6) 
and  their  counterparts  then  establish  the  required  first  partial 


derivatives,  VpJ j  --Equations  (2.7.7)  are  used  to  determine 

the  diagonal  elements  of  V^J  and  i . ,  1=1,..., ,r.  Some  of  the 

•  JL  P  pi 


cross  partlals,  V  J,  V  are  determined  by  applying  equations 

P  P  •  • 

(2.7.6)  to  the  function  J  and  T.  The  other’s,  of  the  form 
J 

■  ,  are  determined  from  the  following  procedure.  If  two  Of 

^1  Pj, 

the  elements  of  p,  say  p.  and  p.,  are  perturbed  by  the  small 
positive  amounts  6p^,  bp^  respectively,  the  corresponding  Solu- 
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tion  of  the  reduced  differential  equations  of  the  extremals 
Mill  be  p^+5p  ).  Taylor’s  series  expansion  then 
gives  ^  1  J  J 


J  + 

0  op 


6p. 


-1- 


i  " 
oPlOp  j  ^i 


3pj^ 

6p  . 


1  a^j 


apj 


(5p^  )‘ 


1 


1  a^j 


,(5pJ^  (2.7.8) 

SpI  ^ 


Substitution  of  equations  (2.7.6)  and  (2.7.7)  into  {2.7.3)  and 
rearrangement  then  gives 


a^j 


J (Pj^-!-6p^  ,p  .j+6p  ^  )+J^- J(pj,  l-6p^ )  -J (Pj+6Pj  ) 

=  ^  ^  (2.7.9) 


Equations  similar  to  (2.7.9)  can  be  found  for  either  or  both  of 
6p  .  and  6p  .  eonsider’ed  negative.  It  is  recommended  that  cross 
paJtials  b^  computed  for  the  perturbations  6p.,  6p  .  taken  in 
the  direction  of  expected  change  in  the  solution.  '^Thls  can  be 
found  from  the  direction  of  steepest  descent. 


Since  one  solution  of  the  reduced  differential  equations 
pf  .tlie_exJ^reij^ls  is  required  to  determine  values  for  the  set  J, 
J,  J,  i’,  T,  one  might  ask  about  the  number  of  solutions  re¬ 
quired  per  iteration  step.  For  the  method  of  steepest  descent 
and  the  fixed  terminal  point  Newton-Raphson  method  this  turns 
out  to  be  2n,  not  counting  the  initial  solution.  The  general 
Nev/ton-Raohson  method  requires  n(n+3)  solutions.  For  small  n, 

say  2  or  3»  and  moderate  computer  solution  time,  this  is  not 
uni'easonable  for  the  general  Newton-Raphson  method.  For  larger 
n  the  computer  time  might  become  excessive.  Under  these  condi¬ 
tions  it  might  be  v/ise  to  use  the  method  of  steepest  descent 
initially  and  finish  off  v/lth  the  general  Newton-Raphson 
method . 


Tvifo  tricks  might  help  when  n  is  large.  If  the,  second 
partial  derivatives  do  not  change  rapidly,  they  need  not  be 
recalculated  every  step.  This  is  especially  true  in  the  vici¬ 
nity  of  the  optimal  point.  The  required  ntunber  of  solutions 
then  drops  to  2n.  Secondly,  one  might  try  a  shift  of  coordi¬ 
nates  to  principle  axes.  This  can  be  accomplished  by  a  trans¬ 
lation,  a  rotation,  and  finally  a  stretch  to  reduce  the  matrix 
of  second  partial  derivatives  to  the  identity  matrix.  Then  the 
cross  partials  disappear  and  the  required  number  of  solutions 
is  again  2n.  An  iterative  method  can  be  constructed  for  this 
process . 


2.8  Example  Problem  I  The  following  simple  analytleal  example 
lilustrates  the  form  (2. 3. 24) -(2. 3. 26)  of  the  solution.  It 
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also  shows  the  necessary  and  sxifflcient  conditions  for  the 
equivalent  minimization  problem.  Ihe  problem  chosen  Is  the 
two  dimensional  harmonic  oscillator  with  a  single  control 
function.  The  problem  was  fovuad  In  reference  (5).  "Ihe  eqxia- 
tlons  of  motion  are; 


^2*  ^l^^o^  "  ^1  <2.8, 1  ) 


+  u,  ^2^^o^  "  '^2  (2.8.  2  ) 

It  is  desired  to  minimize  the  control  effort  over  the  path. 
Ihis  may  be  expressed  in  integral  form  as: 


dx^ 


dXg 

cTT 


J  = 


/ 


u^  dr 


(2.8.3) 


The  terminal  surface  is  specified  by  a  given  terminal  time  and 
by  Xj^(t)  =  0.  Thus, 


=  T-K  =  0  (2.8.4) 

V2  =  T)  =0  (2.8.5) 


vihere  T  is  the  value  of  t  on  the  terminal  surface .  The  Hamil¬ 
tonian  for  this  system  Is 


H  =  p^X2  -i-  u)  u^. 


(2.  8.  6  ) 


The  Euler -Lagrange  equations  are  thus 


Pt  =  Pv 


(  2. 8.  7  ) 
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(2.8.  8  ) 


(  2.8.9) 


The  optlriial  control  function,  from  (2.8..9),  is  seen  to  be 


u  =  - 


(2 . 8 . 10  ) 


This  control  function  automatically  satisfies  the  minimum  prin¬ 
ciple  and  the  Clebsch  necessary  condition.  Eqiiations  (2.8.7) 
and  (2.8.8)  are  linear  equations  v/lth  constant  coefficients, 
and  may  be  solved  directly.  In  terms  of  Initial  conditions, 

r  cos  t  sin  nr^^i°i 


P2(^) 


•  sin  t  cos  tl 


(2.  8.11) 


where  p^  and  p®  are  the  initial  conditions  on  Pj^  and  Pg. 

The  homogeneous  part  of  equations  (2.8.1)  and  (2.8.2)  is 
the  same  as  that  x'or  equations  (2.8.7)  and  (2.8.8).  The  sys¬ 
tem  is  thus  self-adjoint.  The  solution  of  equations  (2.8.1) 
and  (2.8.2)  can  then  be  \\rritten  as: 


( t ) 

X2(t) 


cos  t  sin  t 


-sin  t  cos  t 


cos  t  sin  t 


sin  t  cos  t 


cos  T  -sin  T  0 


sin  T 


P2('r)<3T 


(  2 . 8. 12  ) 


After  substitution  and  integration,  one  finds 


x^(t)' 


Xo(t) 


cos  t  sin  t:  X 


-sin  t  cos  t 


1 

-  Zr  ,  - 


cos  t  -sin  t),t  sin  t  P1 
t  sin  t,(sin  t  t  cos  t)  pS 


(2.8.13) 


v/here  t^  has  been  taken  as  zero.  Finally,  the  solution  of 
equation  (2.3.3)  is 

J(t)  =  ^|p°^(t-sin  t  cos  t)-2,  P°p2  sln^t 

-i  P2^(t  4  sin  t  cos  t)^  .  (2.8 

Equations  (2.8.11),  (2.8.13),  and  (2.8.14)  are  the  solutions 
referred  to  in  equations  (2.3.24)  and  (2,3.25). 

On  the  terminal  surface,  then. 


(2.8. 14  ) 
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r 

J(T)  =  |p°2(T-3ln  T  cos  T)  -  2  pjp®  sln’T 


-i-  P2^(T  -i  sin  T  cos  T 


) 


(2  a  15) 


-X^(T)- 

cos  T  sin  T‘ 

xj- 

_-sTn  T  cos  T_ 

O  OJ 

(T  cos  T  -  sin  T),T  sin  T 
-T  sin  T, (sin  T  +  T  cos 


(2.8.16) 


The  function  F  of  equation  (2.4.6)  for  the  equivalent  minimi¬ 
zation  problem  may  be  i/ritten,  from  (2.8.15),  (2.8.4)  and 
(2.3.5)  as 

(2.8.17)  F  =  J(T)  e^  (T-K)  e2  (t).  (2.8.17) 


The  first  necessary  condition  then  gives 

^  -  0  =  ^(p°  sin  T  -  p°  cos  T)^  -i-  e^ 


+02  {-x°  sin  T  +  x®  cos  T  +  ^[p°  T  sin  T 

-^2  (sin  T  -i-  T  cos  T)j  } 

0  =  |[pj  (T  -sir  ?  cos  T)  -pg  sin^T’] 


oF 

'>  o 
cp^ 


4 


(T  cos  T  -  sin  T) 


(2,8.18) 


(2.8.19) 


Oi< 

SPo 


=  0  =  ^[-pj  sln^T  :  p°  (T  sin  T  cos  T)] 


'2 


T  sin  T 


(2.8.20) 


Tliese,  together  v;ith  (2.8.4)  and  (2.8.5)  are  to  be  solved  for 
I'j  P°>  ^2*  ®i  ®2*  solutions  are 


T-K 

p°  =  A  cos  K 
Pg  =  A  sin  K 

a[x°  K  sin  K  -  Xg  (K  cos  K  -  sin  K)] 
®1  “  (iC  -  sin  K  cos  I'C)  ' 


(2.8.21) 

(2.8.22) 

(2.8.23) 

(2.8.24) 
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02  =  A  =  - 


COS  K  -1-  Xg  sin 


k] 


(K  -  sin  k  os  K) 


(2.8.25) 


provided  K  /  0. 

The  point  defined  by  equations  (2,8.2l)-(2.8,24)  Is  truly 
a  ralnlralzlng  point.  This  v/111  now  be  shoiirh  by  constructing 
the  matrix  A^.  of  equation  (2.4,33).  The  set  (2.8,4)  and  (2.8.5) 
is  first  completed  by  adding  the  equation 

-  ^2  ■:  b  C, 

Then  the  equation  (2.4.29)  may  be  vn'ltten,  for  the  critical 
point  as 

0  0 


(2.8.26) 


vfhere 


'^11 

"21 


X, 


'22 


o' 

0 

— 

0 

)  , 

X 

Xii  = 


OPn 


'12 


o::. 


op^ 


C>Pl 


"22 


ox^ 


op^ 


(2.8.27) 


Equation  (2.4.30)  then  becomes 


1 

0 

0 

=. 

3-21 

ro 

no 

^23 

^2 

^31 

®32 

QrQ  0 

v/here 

L_  M 

®ai  '  ‘ 

i  -'r  V  _ 

^'22.- 

^2), 

^22  " 

1 

^  "22  ”  "21  ^12^  * 


=  CX22,  j  =■ 

-0*12 

^33 

“11 

Also, 


vS  =1 


(2. 8. 28) 


KslnK 


KslnK  ^(K-slnKcosK) 


|-^(  slnK+KcbSK) 


“^(sinK+KcosK) 

-sln^K 


^(K+slnKcosK) 


(2.8  29) 


After  all  the  substitutions  liave  been  made,  it  Is  foiihd  that 
the  matrix  A^^  of  equation  (2.4.33)  is  the  scalar 
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(2  8.  30) 


A4 


(K-slnKcogK) 

4(K^-sln®K) 


Since  K  >  sin  K  for  all  K  >  0,  A^,  >  0,  It  Is  thus  concluded 
that  the  solution  Is  Indeed  a  minimizing  solution,  since  there 
Is  only  one  critical  point. 


As  a  final  comment  for  this  problem.  It  is  noted  that  the 
ralnirnizing  path  Is  liniquely  specified  by  the  constants  IC,  x,  , 
and  x® . 


2.9  Example  Problem  II  The  problem  considered  here  Is  the 
time  optlroal  bang-bang  problem.  The  objective  is  to  shovf  that 
at  least  some  problems  with  corners  can  be  treated  by  the 
methods  described  above.  The  problem  v/111  not  be  solved.  It 
will  only  be  shown  that  the  function  P  of  the  equivalent  mini¬ 
mization  problem  has  partial  derivatives  of  at  least  second 
order  In  the  variables  T,  p  .  It  Is  thus  possible  to  show 
sufficiency  and  to  use  the  ^Nev/ton-Raphson  method. 

The  differential  equations  for  the  time  optimal  bang-bang 
problem  v;lth  a  single  control  fLinction  are  linear,  and  may  be 
vrritten 

• 

X  =  Bx  bu,  5:(t^)  =  x^  (2.9.1  ) 


v;here  b  and  x  are  n;:l  matrices  and  B  is  n;cn.  The  elements 
of  b  and  B  are  constants.  The  control  function  is  constrained 
by  the  equation 

|uj<  1  (2.9.2  ) 

v/hich  may  be  revs'ltten  as 

1  -  u^  >  0  (2.9.  3  ) 

It  Is  recalled  that  inequality  constraints  must  be  In  the  form 
S’  ^  S’  and  that  continuous  partials  of  at  least  third  order 
must  exist.  It  Is  laiovm  that  there  Is  no  optimal  solution 
when  |.u  I  <  1.  'The  function  F  for  the  problem  of  Bolza,  desig¬ 
nated  L  here,  may  thus  be  written 

• 

L  =  1  p(Bx  bu  -  x)  R  (l-u“)  (2.9.4) 


The  Euler -Lagrange  equations  are  then 
-p  =  p  B 


(2.9.  5  ) 


0  =  pb  -  2M>  u. 

Equations  (2.9.6)  and  (2.9.3)  can  be  solved 
provided  the  determinant  R.-  of  eouation  (2.3.31) 
Thus, 


( 2. 9.  6  ) 

for  i-t  and  u 
Is  non-zero. 


det 


-2M. 

-2u 


0 


(  2.9.  7  ) 


is  alwaj'-s  non-zero  since  |  u|“  1.  The.  proper  sign  for  the  con¬ 
trol  function  Is  fovuid  from  application  of  the  minimum,  prin- 
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ciple.  Ilius,  It  is  found  that 

p  bu  <_  ^  U  (  2 . 9.  8  ) 

v^here  U  is  any  admissible  control  fiuictlon.  It  is  concluded 
that 

u  =  -sgn  (2.9.  9) 

Ihen  from  equations  (2,9.9)  and  2.S.6), 

|i  =  -  |1p  F  |.  (2.9.  10  ) 


It  is  noted  that  |J.  is  alv/ays  negative. 

There  are  two  types  of  subarcs  for  this  problem,  corres¬ 
ponding  to  u  =  +1  and  u  =  -1  respectively.  If  u  =  +1,  then 
must  be  negative.  The  reduced  differential  equations  for 
this  arc  may  thus  be  vrcltten, 

X  =  Bx  +  15'  ( 2 . 9. 1 1 ) 

^  _  (u  =  -M) 

P  =  -pB  (2.9.  12  ) 

Furthermore , 

P  =  I  p  (2.  9.  13  ) 


hlien  u  =  -1,  the  dlf fereiitlal  equations  are 
X.  -  Bx  b 

^  _  (u  -  -1) 

p  =  -p  B 

and  the  multiplier  equation  is 


P 


I  P  b. 


(2.9.  14  ) 
(2.  9.  15  ) 

(2.  9.  16  ) 


Now  p  is  at  least  absolutely  continuous,  since  p  is  continu¬ 
ous  and  F  is  constant.  Furthermore,  since  p  <_  0,  it  Is  con¬ 
cluded  that  the  sv/ltchlng  points  iaust  be  the  zeros  of  P. 
ihus,  the  terminal  surface  equation  for  each  subarc  (except 
for  the  last  one)  is 

0  -  (2.9.17  ) 

• 

So  long  as  p  F  0  at  these  points,  the  arguments  of  article 

2.3  hold..  It  is  then  concluded  that  the  solution  is  a  con¬ 
tinuous  function  of  the  initial  conditions,  p.,  and  that  con¬ 
tinuous  partial  derivatives  of  at  least  second  order  In  the 
variables  t,  p  exist.  The  time  optimal  bang-bang  problem 
is  thus  one  problem  v;lth  coi'ners  v/hlch  Can  be  treated  by  the 
methods  given  above, 

2.10  Example  Problem  III  The  problem  chosen  Is  that  of  mlhl- 
rnlsing  the  deviation  of  heating  rate  from  a  specified  heating 
rate  for  a  re-entry  vehicle..  This  corresponds  physically  to 
a  radiating  vehicle  v/hlch  radiates  heat  most  efficiently  at  a 
specified  temperature.  I-la thematically,  the  quantity  to  be 
rainlmlsed  may  be  written  as  an  integral  over  the  path,. 
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T 


dT 


(  2.  10.  I ) 


where 


tg  =  Initial  time 


T  =  terminal  time 

« 

Q  =  specified  constant  heating  rate 

« 

q  =  calculated  heating  rate,  a  f\inction  of  the  path. 

Ihe  unit  of  J(T')  is  heating  rate  squared.  Thus,  if  J(T)  is 
minimized,  then  so  is^J(T^2',  the  heating  rate. 

An  inequalltj'-  constraint  on  the  problem  is  that  the  sensed 
acceleration  never  exceed  a  specified  constant.  This  may  be 
vrritten  as 

(2  .10.2  ) 


ap  <  B 


;/here 


P 


specified  constant 

Instantaneous  sensed  acceleration,  a  func¬ 


tion  of  the  path. 

Tne  i.aodel  chosen  for  this  problem  is  two  dimensional  Vflth 
a  spherical  earth  and  e;q3onential  atmosphere.  The  equations 
of  motion  are; 

p  sin  7 


dt 


2m 


d'~/  „  V 
dt  R 


cos  7  .  S  n  f  1 

(rrfr  P  -  —  (in)  ^ 


n  _  V  cos  7 
dt  (I  T 


(2. 10.  3 ) 

(2.10.  4) 
(  2 .10.  5  ) 
(2.10.6) 


vfhere : 


S  =  reference  area  (constant),  (ft)^ 
m  ?=  vehicle  mass  (constant),  sivigs 
g  =  value  of  gravity  at  sea  level,  ft (sec)” 
R°  =  earth  radius,  ft,_p 
V  -  velocity,  ft(sec)” 

v  =  flight  path  angle,  positive  up,  radians 

•i  h 


dimensionless  altitvide 


h  -  altitude,,  ft* 

C  =  R0,  great  circle  range,  ft.  _3 

p  =  e?qp  (-pRi),  density  in  slug  (ft)” 

p^  =  sea  level  density 

p  -■  atmospheric  exponential  constant  (ft) 


-1 


Tne  aerodynamic  coefficients  are  based  upon  flat  plate  Nevf- 
tonian  flow.  They  may  be  vn’itten  as: 

Cj^  —  Cj^Q  sin  a  cos  a  jsin  a  | 
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S  ^DO  ' 

v/here:  *^DL  constants 

a  -  angle  of  attack,  the  contr'ol  function  of  the 
probleiTi. 

The  heating,  rate  equation  is  assui'iied  to  be  that  given  by 
Ghapiiian:  (4) 


v/hei’e:  c  =  constant. 

It  is  realised  that  equation  (2.10.7)  is  an  approximation  to 
the  convective  heating  rate,  and  that  radiative  heating  plays 
an  important  role  in  the  super-circular  re-entry  region. 
Equation  (2,10.7)  was  chosen  to  simplify  the  overall  problem. 
Finally,  the  pilots*,  or  sensed,  acceleration  is: 


a 


P 


(2.  10.8  ) 


To  completely  specify  the  nature  of  the  solutions,  one 
must  provide  initial  conditions  for  the  dlffer’entlal  equations 
(2 . 10.3)- (2.10.6 )  as  well  as  terminal  conditions.  It  is 
assumed  that  initial  conditions  are  known,  and  that  terminal 
conditions  can  be  I'/ritten  in  the  form: 


I'^(x(T),  T>  =  0,  1  =  1,  ...,  q  <  ntl  (2.10.  9  ) 

v/here:  x(T)  has  components  v(T),  7(T),  i;(T)  and  C(t) 

n  =  number  of  components  of  x(t)  -  4. 

Equations  (2.10.9)  represent  surfaces  in  phase  space,  The 
intersection  of  these  surfaces  is  a  surface  upon  which  the 
terminal  point  of  the  optimal  path  must  lie.  Hence,  equations 
(2.10.9)  are  said  to  describe  the  terminal  surface .  If  q=n+l, 
then  the  problem  is  a  fixed  end-poln't  problem.  IT  q  =  n  then 
the  terminal  surface  is  a  curve.  For  q  <  n,  (2,10.9)  describes 
a  sm-’face  in  phase  space. 

Initial  conditions  for  this  problem  are  assumed  to  be  in 
the  super-circu-lar  velocity  region.  The  tei^rainal  surface 
equations  are 

i'l  =  Sf  -  T  =  0  (2.  10.  10) 

^'2  "  ^2  "  °  (2.  10.  II  ) 

=  a^  -  7(T)  =  0  (2,  10.  12) 

’^4  =  a4  -  ^(T)  =  0  (2.  10.43) 


a^  -  C(T)  =  0 


(2  .  10  14  ) 


v/here  the  constants  a^,  a.^,  a^r  a^^  and  a^  are  specified  values 
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In  the  sub-circular  velocity  region.  Various  combinations  of 
equations  (2.10.10)-(2.10.14)  can  be  used  to  obtain  different 
terminal  surfaces,  and  hence,  different  optimal  solutions. 

The  problem  may  now  be  stated;  Find  that  programming  of 
the  control  function,  a(t),  such  that  the  solution  of  the  dif¬ 
ferential  equations  (2. 10. 3) -(2.10. 6)  for  given  Initial  condi¬ 
tions  satisfies  the  inequality  constraint  X2.10.2),  ends  on 
the  terminal  surface  defined  by  the  chosen  combination  of 
equations  (2.10.10)-(2.10.l4) ,  and  yields  the  minimum  value 
for  the  function  (2.10.1)  (if  a  solution  exists). 

To  facilitate  the  problem  solution,  a  new  variable  will 
be  defined  by  the  differential  equation 

=  (Q-4)^.  x^(t^)  =  0  (2.10.15) 

This  is  added  to  the  set  ( 2. 10. 3) -(2. 10.6 ) .  Equation  (2.10.1) 
may  then  be  written 

x^(T) 

J(T)  =  ^  .  (2.  10.  16) 

o 

The  solution  Is  assumed  to  consist  of  two  types  of  subarcs, 
one  for  which  a  <  B  (except  for  possibly  a  finite  number  of 
points)  and  the^other  foi'  which  a  =  B.  The  function  P  of  the 
multiplier  rule  can  then  be  v/rltten  In  the  abbreviated  form 

F  =  A^(f^-v)  -I  A2(f2-T)  -i-  A2(f3-^)  -I  ''-4(^4-^) 

-l-X-(f  3-X3)  +  M-  •  (B-ap  ) ,  (  2.  10  17  ) 

where  It  Is  understood  that  the  multiplier  M-'  Is  zero  whenever 
a  <  B,  and  <  zero  when  a  —  B.  The  coefficients  of  the  multl- 
pSlers,  X.,  are  the  differential  equations  (2.10. 3)-(2. 10.6) 
and  (2.10.15).  f..,  1=1  i  •••>  5>  represents  the  right-hand 

sides  of  these  dlrferential  equations.  The  Euler -Lagrange  equa¬ 
tions  for  (2.10.17)  can  then  be  written 


dX^ 

■  dl“  ^ 

"2 

af2 

^^3 ... 

dv 

■  H 

af4 

dft- 

+  ; 

'^5  ^ 

(2. 

10. 

18  ) 

dX2 
■  dt 

^1 

+ 

67 

X2 

^^2  ,  ^ 
3T  '  3 

afo 

■  ^^4 

Sf4 

(2. 

10. 

19) 

6f, 

ofo 

Ofi, 

dXg 

"  dt  "" 

^1 

1  + 
5T 

X2 

2 

1 

■  '^4 

cTX 

afr 

oa 

.1- 

'^5 

wi  - 

(2 

10 

20) 
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if 

!! 

0 

dXc 

^  r- 

5 

^  ^2 

6f, 

c)fo 

oa 

h  5^ 

^2  ^ 

H'  ^ 
da 

(  2.  10.  21 ) 

(  2.  10.  22) 

-  0  12.  10.  23) 


The  multiplier  of  equation  (2.10.21)  is  a  constant  because 
of  the  original  choice  of  coordinate  system  for  the  dif¬ 
ferential  equations.  Polar  coordinates  always  give  a  cyclic, 
or  ignorable,  coordinate.  This  is  the  angle  &,  which  never 
appears  on  the  right-hand  sides  of  the  differential  equations. 
In  Cartesian  coordinates,  there  would  be  four  variable  multi¬ 
pliers  Instead  of  three .  It  is  advisable  to  use  that  coordi¬ 
nate  system  which  yields  the  greatest  number  of  cyclic  coordi¬ 
nates.  The  corresponding  multipliers  are  constants.  Xj-  is  a 
constant  because  of  the  definition  of  x^.  Its  value,  ^  from 
equation  (2.10.16)  and  the  transversality  condition  (2.3.13)i 
is  found  to  be 


1 

T-t  • 

o 


(2. la  24  ) 


This  follows  since  >!^ir(T)  is  not  present  in  the  terminal  surface 
equations  (2.10.10)-^(2 .10.14)  and  since  J  =  g  in  equation 
(2.2.1).  The  Hamiltonian  for  this  problem  can  be  v/ritten 


5 

H  =  ^  Pl^i 

0 


(  2.  10.  2  5  ) 


It  is  a  constant,  and  its  value  (the  right-hand  side  of 
(2.10.25))  is  fo\md  from  equations  (2.10.16),  (2.10.10)  and 
the  transversality  condlton  (2.3.12).  The  constant  el  is  zero 
if  equation  (2.10.16)  is  not  present  for  the  problem  "^xinder 
consideration.  Finally,  terminal  values  for  the  other  multi¬ 
pliers  are  seen  to  be 

X^(T)  =  e^  (2.  10.  26) 

>.2(1)  =  e^  (  2  .  10.  27 ) 

.^.^(T)  =  e^  (2.  10.28) 


44 


(2 .  10. 29  ) 


Any  one  or  all  of  these  are  zero  if  the  corresponding  equations 
(2.10.1l)-(2.10.l4)  are  absent  from  the  problem  formulation. 

It  is  convenient  to  normalize  the  multipliers  at  this 

point .  This  v-Zill  be  done  by  multiplying  each  multiplier  by 

the  time  interval,  T-t  ,  viewed  as  a  constant.  Thus 

o 
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Pi  =  (T-to)  Xi,  1=1,..., 5 

Sj  =  (T-to)  ej,  J=l,...,r 


P.  -  (T-t^y  H- 


i  LL  • 


(2.  10.  30) 
(  2.  10.  31  ) 
(  2  .  10.  32) 


Then  equations  (2.10.l8)-(2,.10.20) ,  (2.10.23)  ,  and  (2.10.,24). 
(2.10.29)  become 

Sf.  ar  ar. 

~^1  ^1  av  ‘  ^2  dv  '  ^3  dv  ^  ^4  dv 


dfp.  da 
ov 


(2.  10.  33) 


df 


df 


df. 


df, 


“^2  “  Pi  ^  ^2  5^  P3  P4  3^  ‘2-  10.  34) 


df  df 

■P3  "  Pi  dT  P2  dT 


Pi 


df^ 


o  <; 


df  c  da 

..-5  .  ,s  ^ 

o  ^  d  s 


of 


0 


Pi 


1  , 


df 


9 

-  P  T-^ 

oa 


H 


1  da  ‘^2  oa 
5 

1  ‘  I  Pi'i " 

■1  '  1 

1-1  y  ....  y-i 


1=1 

Pi(T)  »  ^ 

Ps  “  1 .' 


(2.  10  3  5  ) 

(2.  10. 36) 

(2.  10.  37  ) 

(  2  .  10.  38  ) 
(  2.  10.  39) 


Co.mputatlonal  equations  v/111  no'o  be  developed  for  the  con¬ 
trol  fimction,  a,  and  the  multiplier  p.  Consider  first  those 
subarcs  for  v/hlch  a  <  B.  Over  these  subarcs  p  =  0.  Then  equa¬ 
tion  (2.10.36)  becoMes 

df 
P 


1  oa 


1  P^2 

P2  dd“  " 


or 


Sfi  acp 
=1  3cJ  S" 


Po  7: 


dfp  dc^ 


2  dC.j.  da 


0. 


Prom  the  aerodynamic  coefficients  one  finds 

II 

-r—  =  3C.r  sin  a  cos  a  sin  a, 
oa  DL  I  I 


(2.  10.40) 

(2.  10.41) 

(2  .  10.  42  ) 
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(2,  10.43) 


'LO 


sin  a 


(3  cos^  a  -  1 ) . 


and 


P 

P 


(2.  10.  44) 


(2.  10.  45) 


.from  equations  (2.10.3)  and  (2.10.4).  Substituting  these  into 
(2.10.41)  and  rearranging  then  gives 

p  V  |sln  c;|  [-3  V  sin  ct  cos  a 


~  ^^1  <2  -10.  46) 

Assux.ilng  re-enti’y  conditions,  the  leading  terms  of  (2.10.46) 
are  non-sero  if  j  sin  a  i  /  0  .  Ihe  bracketed,  terms  may  then  be 
rewritten  in  the  form 


tan  a 


3^DL  Pl^ 
‘^LO  ”2 


tan  a 


2-0. 


One  is  thus  led  to  the  equation 


tan  a  =  -a  -I  -vA 


(2.10.47) 


(2.  10.  48) 


v/here 


a  = 


-'^DL 
2U 


LO  ^2 


It  is  noticed  that  the  sign  of  tan  a  is  determined  by  the  choice 
of  the  i  sign  in  equation  (2.10.48). 

The  lift-drag  polar  is  traversed  once  as  a  ranges  through 
Tf  radians.  It  is  thus  advisable  to  limit  the  control" function 
to  a  range  of  tt  radians  to  avoid  double  values.  Furthermore, 
the  points  a  0,7t  are  singular  points  since  d^F  (the  deter- 

dd^ 

ralnant  of  equation  (2.3.21)  is  zero  at  these  points.  The  range 
0  <  d  <  7r  is  chosen  here  since  then  |  sin.  dl  =  sin  a,  and  the 
proper  sign  for  equation  (2.10.48)  is  most  easily  chosen.  The 
singular  points  are  removed  by  defining  a  =  0  at  these  points . 

Thus,  the  range  of  a  is 

0  <  d  <  TT  .  (2.10.  49) 

Application  of  the  minimum  principle  gives  the  equation 
sin  d  Pj^v  sin^d  +  Cj^q  Pg  slnd  cosdj 

£  sin  A  p^v  sin^A  i  Cj^q  p^  sinA  cosa]  (2.  10.50  ) 

where  A  is  any  value  of  the  control  function  satisfying  eqiia- 
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c 


tlons  (2.10.49)  and  (2.10.2).  Substitution  of  (2.10.47)  into 
(2.10.50)  and  reari’anging  then  gives 

P2  sina  tana  sin  A  tanA  (2.  10.  51) 

Since  sina  ^  0  and  the  sign  of  tana  is  determined  by  the  -  sign 
of  equation  (2.10.48),  it  is  seen  that  (2.10.51)  is  satisfied 
by  the  choices 

If  Pg  >  0,  choose  -  sign  (^  <  a  <  tt) 

If  P2  <  0,  choose  ir  sign  (O  <  a  <  ^) . 

Equation  (2.10.48)  thus  becomes 

tan  a  =  -a  -  ( sgn  P2 )  -i  2’  (  2 .  lO.  52 ) 

The  behavior  ivhere  P2  =  0  v;ill  nov/  be  er.amined  .  This  uill 
be  done  as  a  limiting  process,  since  p..  ,  P2  and  v  ai'e  continu¬ 
ous  fxmctions  of  time.  The  binomial  series  expansion, 

assuming  large  a,  gives 


It  is  seen  that : 

If  a  >  0  and  the  minus  sign  v;ere  chosen,  then 

tan  a  =  11m  -2|a|,  a  -5>  ^  (from  the  left). 
a->oo 

If  a  <  0  and  the  plus  sign  chosen, 

tan  a  -  11m  2|a|  ,  a  ->  ^  (from  the  right). 

-  a->oo 

On  the  other  hand,  if  a  <  0  and  the  minus  sign  chosen, 

tan  a  =  lj.m  -  -  0,  a  ->  (from  the  right). 

-  a->cp  rl 

Finally,  v/lth  a  >  0  and  the  plus  sign, 

tan  a  -  lim  -ri-r  =  0,  a  ->  0  (from  the  left). 

a->aDrl 

The  last  two  cases  shov/  that  it  is  possible  foi’  a  to  be 
discontinuous.  For  example,  if  P2  passes  through  zero  and  a 
v;as  near  'n ,  then  a  v;ill  Jump  to  zero  as  Pp  goes  through 
zero.  This  difficulty  is  cleared  up  if  the  proper  interpreta¬ 
tion  is  made.  Since  tan  (7r  a)  =  tan  a,  c;  may  be  thought  of 
as  being  continuous  across  Jumps,  but  in  the  disallowed  region 
7r  <  a  <  277.  The  control  function  discontinuity  V7ill  be  re- 
taTned  since  computer  results  can  be  properly  interpreted  and 
since  it  allov/s  |sin  aj  =  sin  a  in  all  equations.  Finally, 
although  of  little  practical  importance,  if  p,  and  Pp  .are  simul¬ 
taneously  zero  at  a  point,  the  limit  ratio  p, 7pp  is  p, /Pq  at 
that  point  by  L'Hospital *s  rule .  did 
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To  suni  up,  the  reduced  differential  equations  of  the 
extremals  for  subarcs  having  a  <  B  are  (2 . 10 .3 )- (2 . 10 .6 ) , 
(2.10.15)  and  (2.10.33)-(2.10.55)  v;lth  (J-  =  0.  The  control  func¬ 
tion  Is  computed  from  equation  (2.10.52)  In  the  range  0  <_  a  <  iv. 

T'he  subarc  terminal  surface  is  seen  to  be  a  =  B,  provided  that 

" 


\Ihen  a  =  B,  one  must  calculate  both  a  and  M-.  The  method 
chosen  for  ^doing  this  here  is  to  find,  a,  from  a  -  B  and  \x  fi'oni 
(2.10.33),  E'cpanslon  and  I'eairaricgement  of  the  ^“’constraint 
equation  a  =  B  gives: 


(2.10.53) 
v/here : 


b^ 

b 


ego  =  sln- 
Spv~ 

p2  ri2 

^DL  "  "LO 


g[c^  sin-’  a 


sin  a 


(  2.10.  53) 


=2  "  ^LO 


p  r 
-  "DO  DL- 


The  right  hand  side  0:'  (2.10,53)  is  a  function  only  of 
the  vehicle  aerodynamic  coefficients  (constants)  and  sin  a.  It 
is  xero  when  a  ~  0  or  ■/, ,  and  majclmuin  v/hen  a  -  ^  .  From  this  it 
follows  that  b  must  satisfy 


(2.10.5^0 


<  b  <  (C 


"DO  -  - 


DL 


"DO 


(2.  10.  54) 


Physically  speaking,  ■  if  >  b  at  any  point  of  the  path,  then 
a  has  gone  to  .:ero  in  the'^  futile  attempt  to  keep  a  -  B.  The 
path  must  then  be  disqualified  as  a  candidate  for  the  optlmuin 
path . 


Newton's  method  vms  chosen  to  extract  a  from  (2.10.53). 

The  iteration  equation  is 

(b^-OS^)  -  sln'^a  fc,  sin"a  -1  c^sin  a  Co] 

(a. 10. 55)  A  a  =  - f- - Ll - ^ - 2 - 12,0  55, 

sin  a  cos  ct  [:6c^sin'"ci  •:  TCgSln  a  3Cg  J 

A  Aa  is  calculated  from  (2.10.55)  using  an  assumed  d  (usually 
the  last  integration  step  value),  This  is  added  to  a,  and  the 
nevf  a  is  used  to  calculate  a  nev/  Aa.  The  process  continues 
until  Aa  is  negligible . 


TT 


In  the  vicinity  of  a  =  0,  ^  the  denominator  of 

(2,10.55)  is  likely  to  be  quite^small.  To  avoid  this  difficulty 
equation  (2.10.53)  was  expanded  as  a  series.  The  results  are: 


(2.10.56) 


If  a-0,  (sin  a 


a"'  \  DOl 

—  t)hGn  CL  ~  i  ~  V  ~~  "  — ! 

^  •  * 


'3  '  / 


(2. 10.56): 
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(2.  10  57) 


If  o~tr,  (flin  a  ■  (tt-o)  -  ) 


then  a 


TT  - 


b*-C« 


DO 


_jl/3 


If  (cos  o  -  -  a)  -  (J  -  a)®)  (2.  lo.  58) 


then  a 


it 


«!dL  ^ 


DO^ 


1/2 


where : 


'^DL^So^  "®L0  J 

vise  +  sign  If  Pg  >  0 
vise  -  sign  If  P2  <  0- 


If  Pg  =  0,  choose  sign  so  that  tan  a  Is  contlnuovis.  The 
choice  of'^slgns  again  comes  from  the  mlnlmvim  principle . 

Finally,  M'  Is  calcvilated  from  equation  (2.10.36)  In  the 

form: 


BET  P2  35“J  • 


Now  may  be  written  In  the  form 


(2.  10.59  ) 


da 


IS  3a~  ^  °L  35~J 


+  C, 


(2.  10  60  ) 


where ; 


1 

S  3a“  35“  “  2 


a  [Scj^sln^a 


+  4CgSln  a  +  Sc^li 


(  2  .  10. 6 1  I 


This  Is  proportional  to  the  denominator  of  equation  (2.10.55). 

It  Is  also  noted  that  the  determinant  of  equation  (2.3.31) 

Is 

(  2  .  iO.  62  ) 


^2  “'Oq 

Since  this  must  never  be  zero  when  a  =  B,  It  is  seen  that  a 
must  never  be  Q,  tt  or  tt.  P 

? 

The  reduced  (llfferentlal  equations  of  the  extremals  for 
the  subarcs  for  which  Sp  =  B  are  equations  (2.10.3)-(2,10.6), 
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(2.10.15)  and  (2.10.33)-(2.10.35)  with  ^  £  0.  The  control  func¬ 
tion  Is  determined  from  the  Iterative  equation  (2.10.55)  or  the 
small  angle  equations  (2.10.56)-(2.10.5o)  In  the  range  (2.10.49), 
except  a  ^  0,  7t  ,  The  multiplier  M>  Is  determined  from  equation 
2 

(2.10.59).  It  Is  easily  seen  that  M-  Is  continuous,  and  that 
the  suharc  terminal  3\irface  Is,  consequently,  =  0,  provided 
tA  7^  0. 

Potential  corner  points  will  now  be  examined.  First,  the 
points  a  =  0,  TT  must  be  ruled  out  as  candidates  for  these 
S’ 

points  since  both  R,  and  R^,  equations  (2.3.21)  and  (2.3.31), 
must  be  non-singular  at  potential  corner  points.  Then  for  all 
other  values  In  the  range  (2.10.49)  It  turns  out  that  a  must  be 
continuous  at  these  points.  This  Is  seen  most  easily  by  look¬ 
ing  at  the  Heimlltonlan.  It  Is  a  continuous  function  of  x,  p 
and  a.  Since  x  and  p  are  continuous  at  potential  corner  points, 
then  a  must  be  contlnuoiis  at  those  points.  Similarly,  examina¬ 
tion  of  equation  (2.10.36)  shows  that  p.  must  be  continuous  at 
those  points.  Thus,  M-  mxist  start  and  end  with  the  value  zero 
on  those  subarcs  for  which  Sp  =  B. 

Finally,  from  sections  2.3  and  2.4,  It  Is  seen  that  this 
problem  cam  be  solved  as  an  Initial  value  problem. 

2.11  A  Special  Form  of  the  Newton-^phson  Method.  Several 
optimal  trajectories  have  been  obtained  for  the  above  problem 
using  the  Newton-Raphson  method.  The  terminal  surface  for  all 
of  these  has  been  described  by  equation  (2.10.10).  For  this 
terminal  surface  only,  the  Newton-Raphson  method  reduces  to  an 
especially  nice  form.  This  will  now  be  developed. 

The  function  P  for  the  equivalent  minimization  problem 
with  the  constraint  eqxiatlon  (2.10.10)  can  be  written 

P  =  J(T,  p)  +  ej^(a3^  -  T) .  (  2.  II.  I  ) 

Ihe  values  T,  T,  p,  p  are  defined  as  In  section  2.5.  Then 
using  equation  (2.5.2)  and  the  first  necessary  condition  for 
the  equivalent  minimization  problem,  one  finds 

^  =  0  =  J  +  (0-p)  -  e,  (2.  II.  2) 

ST  P  ^ 

VaP  »  U  »  7pJ  +  V*J  (0-p)  (2.  11.  3  ) 

0  -  -  T  (  2.  II.  4  ) 

These  are  partially  solved  by 

T  “  (2.  II.  5 ) 

ej_  -  j  +  Vpj  (p-p)  .  (2.  11.6) 

The  problem  then  reduces  to  solving  (2,11,3)  for  p  subject 
to  the  sxifflciancy  condition  (2.4.21). 


Ihe  sufficiency  condition  will  now  be  reduced  to  the  equi¬ 
valent  from  (2.4.331  by  conq)letlng  the  set  of  constraint  equa¬ 
tions.  Ihe  constraint  equations  need  only  be  Independent  func¬ 
tions  of  T  and  p.  Consequently^  the  eqxiatlons  (2.4.23)  way  be 
chosen  as 

p  ^  p  +  b  7  (2.  11.7  ) 

Then  equation  (2.4.30)  becomes 


where  I  Is  the  (rH-l)x(n+l)  Identity  matrix.  Substitution  of 
(2.11.8)  Into  (2.4.31;  then  gives,  for  the  form  (2.4.33), 

J  ^  I  0*  (2.  II.  9  ) 

It  Is  thus  seen  that  the  matrix  V®J  rtlvust  be  positive-definite. 
The  special  form  of  the  Newton-Raphson  method,  then,  consists 
of  the  Iterative  solution  of  equation  (2.11.3)  provided  the 
matrix  V^J  of  that  eqiiatlon  Is  positive-definite. 

2 . 12  Experiences  In  the  Use  of  the  Newton-Raphson  Method  The 
problem  of  section  2.10,  as  stated  above,  has  been  solved  using 
the  special  form  of  the  Newton-Raphson  method.  The  overall  per¬ 
formance  of  the  method  has  been  Very  satisfactory.  Once  the 
Initial  values,  p,  are  close  enough  to  the  minimizing  values, 
p,  the  method  converges  quite  rapidly  to  the  minimizing  point. 

There  are  certain  problems  involved  In  the  use  of  the 
Newton-Raphson  method,  as  there  are  with  any  other  optimizing 
scheme.  The  first  of  these  Is  the  nvunber  of  solutions  of  the 
reduced  differential  equations  of  the  extremals  required  per 
Newton-Raphson  step.  As  stated  In  section  2.7,  n(n+3)  solu- 

2 

tlons  are  required.  Gradient  schemes,  (6,8)  on  the  other  hand, 
require  on©  forward  and  one  backward  solution  per  step.  However 
the  convergence  properties  of  the  Newton-Raphson  method  theo¬ 
retically  become  better  and  better  as  the  minimum  point  Is 
approached,  whereas  gradient  schemes  tend  to  blow  up.  It  Is  an 
open  question,  then,  as  to  whether  one  method  requires  more 
total  computer  time  than  the  other.  Furthermore,  one  Is  never 
certain  that  a  relative  minimum  has  been  obtained  using  grad¬ 
ient  techniques. 

The  biggest  problem  Involves  the  Integration  accxiracy. 

Since  the  partial  derivatives  are  approximated  by  differencing 
Integrated  solutions,  the  random  error  generated  by  the  Inte¬ 
gration  algorithm  may  result  In  bad  predictions  of  the  minimiz¬ 
ing  point.  Ihe  value  of  the  Hamiltonian,  theoretically  a  con¬ 
stant,  may  be  used  as  a  measure  of  the  goodness  of  the  solution. 
It  should  be  remembered  that  any  Integration  algorithm  will 
blow  up  after  a  sufficient  number  of  Integration  steps. 

Closely  related__to  the  above  problem  Is  that  of  picking 
the  perturbations,  6p,  of  article  2,7.  If  these  are  too  small. 
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the  partial  derivatives  will  reflect  integration  error  alone. 

If  they  are  too  large,  the  partial  derivatives  will  not  reflect 
the  nature  of  the  surface  In  the  vicinity  of  the  point  T',  p  . 
Intermediate  values  must  be  picked  from  experience,  ° 

Solution  of  the  problem  of  article  2.10  was  orlgiiwlly 
attempted  using  Bryson’s  method  of  steepest  descent. The 
terminal  surface  In  this  case  was 


This  equation  describes  clrcxilar  velocity  at  a  given  altitude. 
Hie  problem  turned  out  to  be  extremely  sensitive  for  this  termi¬ 
nal  surface.  Nevertheless,  one  trajectory,  thought  to  be  fairly 
close  to  optimum,  was  obtained.  Vfhen  the  Newton-Raphson  method 
was  used  for  the  same  problem,  the  steepest  descent  value  of  J 
(equation (2.10,1) )  was  halved  before  the  sensitivity  problem 
became  severe.  Furthermore,  the  amount  of  human  Intervention 
required  for  the  Newton-Raphson  method  was  much  less  than  that 
for  the  method  of  steepest  descent.  It  Is  the  writer’s  opinion, 
considering  the  problems  Involved  with  both  methods,  that  the 
Newton-Raphson  scheme  Is  by  far  the  more  superior  method. 
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INTRODUCTION 

This  paper  describes  some  of  the  results  of  a  current  research  pro¬ 
ject*  to  apply  optimal  control  techniques  to  the  design  of  flight  control  sys¬ 
tems.  The  project  will  be  completed  in  October  1962,  and  a  final  report 
containing  the  complete  findings  will  be  published  soon  thereafter. 

The  area  of  optimal  control  theory  of  primary  interest  in  this  study  has 
been  the  optimal  regulator  problem  for  a  constant  linear  plant  with  a  per¬ 
formance  index  extended  over  an  arbitrarily  large  time  interval.  The  opti¬ 
mal  control  for  this  problem  is  a  set  of  feedback  loops  with  constant  gains  if 
the  control  input  amplitude  is  not  constrained.  (Although  the  control  is  not 
restricted,  the  integral  of  control  input  squared  is  minimized.)  This  solution 
is  of  interest  to  flight  control  because  constant  feedback  gains  are  easy  to 
mechanize  and  because  computer  solution  of  the  optimal  regulator  problem 
under  these  conditions  is  straightforward  (non-iterative).  It  is  also  of  inter¬ 
est  as  a  possible  technique  for  automatic  digital  computer  synthesis  of  multi¬ 
input,  multi -output  linear  systems  of  high  dimension. 

The  paper  briefly  outlines  the  theory,  discusses  the  flight  control  ap¬ 
plication  in  general  and  then  presents  some  results  on  two  specific  problems: 
1)  the  problem  of  obtaining  adequate  lateral  handling  qualities  in  an  X- 15- 
type  aircraft,  and  2)  the  attitude  control  problem  in  large,  flexible  boosters. 

The  optimal  regulator  probleni  for  constant  linear  plants  has  been 
solved  by  S.S.L.  Chang  and  others  for  the  single -input,  single^output  case. 

R.  E.  Kalman,  at  the  Martin  Company's  Research  Institute  for  Advanced 
Studies,  has  solved  the  general  linear  case  with  a  constant  or  time-varying 
plant.  The  authors  are  indebted  to  Dr,  Kalman  for  his  advice  during  this 
program  and  to  RIAS  for  the  use  of  their  IBM  7090  computer  program. 


THEORY 1 

A  constant,  linear  plant  in  state-vector  form  is  represented  as: 

^  =  Fx  +  Gu  (1) 

dt 

^  Sponsored  by  the  Flight  Control  Laboratory,  Aeronautical  Systems  Division, 
USAF  -  Lt-  Edwin  Stear,  ASD  Project  Engineer. 
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where  x  is  the  n- dimensional  state  vector  of  the  plant,  u  is  the  m -dimensional 
control  vector,  and  F  (dimension  nxn)  and  G  (dimension  nxm)  are  constant 
matrices.  There  are  no  constraints  on  x  or  u. 


Define  the  output  of  the  plant  as 

y  =  Hx  (2) 

where  y  is  a  p-dimensional  vector  and  H  is  a  constant  matrix  of  dimension 
pxn.  (The  output  can  have  more  than  one  interpretation  -  vector  of  measur¬ 
able  quantities,  error  vector  for  the  regulator  problem,  arbitrary  linear 
function  of  the  state,  etc.). 


Define  a  performance  index  as 


2V  =  L 

T-^oo 


:(T),Sx(T)+  y*  [y(r),  Qy(r  )  +  a(T),  Ru(r)]dr 
t 


(3) 


where  t  =  present  time 

T  =  terminal  time 

S  =  non-negative  definite  symmetric  matrix  (nxn) 

Q  =  positive  definite  symmetric  matrix  (pxp)  . 

R  =  positive  definite  symmetric  matrix  (mxm) 

and  the  notation  a,  Aa  denotes  the  scalar  product  of  the  vectors  a  and 
and  Aa.  The  forms  x(T),  Sx(T)  and  y(T),  Qy(T)  and  u(7r),  RuCT)  are  quadratic 
forms  in  the  terminal  state,  the  output  and  the  control  input  respectively. 


The  optimal  regulator  problem  is  the  problem  of  finding  u  as  a  func¬ 
tion  of  X  such  that  the  plant  moves  from  some  initial  x  to  the  point  x  =  0  by 
the  path  that  minimizes  V,  This  problem  was  solved  by  Kalman^  using  the 
Hamiltonian  calculus  of  variations  and  the  work  of  Caratheodory.  The  re¬ 
sult  is  briefly  stated  as  follows: 


Assuming  that  a  minimum  of  V  exists  and  that  it  can  be  written  as: 

2Vmin  =  x(t).  Px(t),  (4) 

we  have 

UoptU)  =  -Kx(t)  =  -R-^G'Px(t).  (5) 

P  is  the  positive -definite  symmetric  solution  of  the  matrix  equation 

F'P  +  PF  -  PGR'^G'P  +  H'QH  =  0  (6) 

fv  satisfies  the  Hamilton- Jacobi  differential  equation,  and  equation  (6)  is 
the^sl^ady  state  of  the  matrix  Riccati  equation, 

— =  F'P  +  PF  -  PGR'^G'P  +  H'QH,  that  results  when 
dt 

^min  substituted  into  the  Hamilton- Jacobi  equation.  ] 

Kalman  has  shown  that  the  sblutioh  exists  whenever  ^le  plant  is  cotq- 
pletely  controllable,  this  quality  beihg  defined  as  the  ability  of  finite  control 


inputs  to  return  the  plant  from  any  initial  condition  to  the  origin  in  finite  time. 
The  solution  is  unique  if  the  system  is  completely  observable,  this  quality 
being  defined  as  the  ability  to  determine  the  state  vector  at  present  time  by 
observing  y  and  its  past  values.  It  should  be  emphasized  that  these  are  suf¬ 
ficient  conditions  for  existence  and  uniqueness.  Solutions  have  been  computed 
even  when  the  plant  lacks  controllability  and  observability.  However,  the 
uniqueness  of  these  solutions  has  not  been  investigated. 

Substituting  for  u  in  equation  (1),  we  find  that  the  closed-loop 
optimal  system  is  described  by  the  equation 

^  =  (F-GK)x  s  Fx  (7) 

This  system  is  asymptotically  stable  iin  the  large,  since  y^nin  Lyapunov 
function. 

Some  examples  of  optimal  regulators  are  illustrated  in  Figure  1. 

These  examples  are  all  single -input,  single -output,  second- order  plants. 
Therefore  the  Q  and  R  matrices  in  the  performance  index  are  scalars  denoted 
as  q  and  r.  Since  one  of  the  scalars  can  be  factored  outside  the  integral  sign 
in  equation  (3),  there  is  only  one  parameter,  q/r,  that  expresses  the  criter¬ 
ion  of  performance.  By  increasing  q/r,  we  more  heavily  weight  errors  in 
output  over  control  input  action.  Figure  1  shows  the  loci  of  the  closed-loop 
poles  as  q/r^s  increased.  These  plots  can  be  generated  in  two  ways.  First, 
the  roots  of  F  can  be  computed  for  each  q/r  by  computing  P  (and  thus  K)  from 
the  Riccati  equation.  Secondly,  the  work  of  Chang^n  single-input,  single  ¬ 
output  systems  (root- square-loeus  technique)  can  be  applied.  The  right-half 
plane  images  of  the  plant  poles  and  zeros  are  added  to  ^e  actual  plant  open- 
loop  roots,  and  a  standard  root  locus  plot  is  constructed.  The  left-half 
plane  poles  of  this  root  locus  plot  are  then  the  closed-loop  poles  of  the  opti¬ 
mal  system.  The  performance  parameter  q/r  is  related  to  the  gain  param¬ 
eter  of  the  plot.  (The  actual  feedback  parameters  are  represented  by  a  ma¬ 
trix  of  loop  gains. ) 

Using  this  second  technique,  it  is  relatively  easy  to  examine  the  char¬ 
acteristics  of  optimal  single-input,  single-output  systems.  It  is  readily  ap¬ 
parent  how  the  optimal  system  will  behave  as  a  function  of  the  performance 
index  being  minimized.  At  this  point  we  can  say  that: 

1.  Not  every  stable  linear  system  optimizes  a  quadratic  index 
of  the  form  of  equation  (3).  The  process  of  optimization 

is  truly  selective. 

2.  Not  every  optimal  system  is  of  high  bandwidth  and  adequate 
damping  (witness  the  pole  positions  for  low  q/r). 

3.  As  q/r  oo  the  optimal  closed-loop  poles  not  cancelled  by 
zeros  approach  a  Butterworth  configuration  (poles  sym¬ 
metrically  arranged  on  a  semicircle  whose  center  is  the 
origin). 
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GENERAL  REMARKS  ON  FLIGHT  CONTROL  APPLICATION 


The  equations  of  motion  of  an  aerpspace  vehicle  are  nonlinear.  How¬ 
ever,  the  motions  of  these  vehicles  can  usually  be  adequately  described  by 
linearized  equations  if  the  perturbations  from  equilibrium  are  small.  If  the 
equilibrium  conditions  change  due  to  changes  in  altitude,  airspeed,  gross 
weight,  or  other  similar  flight  parameters,  the  coefficients  in  the  linearized 
equations  also  change.  If  it  carl  be  assumed  that  the  vehicle  will  not  stray 
very  far  from  some  precalculated  flight  trajectory,  then  these  coefficients 
can  be  specified  by  functions  of  time  only.  Frequently  these  coefficients 
change  so  slowly  that  they  can  be  considered  constant.  It  is  therefore  evi¬ 
dent  that  the  plant  in  this  case  fits  the  general  linear  optimal  control  theory 
and  often  the  theory  for  a  constant  linear  plant. 

However,  for  optimal  control  to  be  meaningful,  we  must  attach  some 
physical-meaning  to  the  performance  index.  The  error  portion  of  the  index 
is  not  difficult  to  justify.  In  the  attitude  control  problem,  we  would  like  to 
minimize  errors  in  attitude  from  some  reference.  In  the  handling  qualities 
problem,  we  have  a  set  of  desirable  closed-loop  dynamic  characteristics. 

We  can  represent  these  characteristics  by  a  model.  Thus  using  Kalman's 
model  formulation  of  the  performance  index  described  below,  we  can  effec¬ 
tively  minimize  the  errors  between  model  and  system  outputs.  The  second 
piece  of  the  performance  index  requiring  minimum  control  inputs  is  not  as 
easily  inte  preted.  In  many  applications,  we  don't  really  care  to  minimize 
control  input  as  long  as  the  control  input  and  rate  limits  are  not  exceeded. 
Minimizing  control  deflections  helps  to  accomplish  this,  but  the  requirement 
kDf  minimization  is  basically  too  strong.  We  do  not  desire  minimum  control 
input  only  input  smaller  than  a  given  level. 

The  real  weakness  in  the  performance  index  as  a  meaningful  criter¬ 
ion  of  performance  is  the  fact  that  the  choice  of  relative  weighting  between 
the  output  error  and  the  control  input  is  arbitrary.  It  has  already  been 
pointed  out  that  the  closed- loop  poles  are  sensitive  functions  of  the  param¬ 
eter  q/r  for  a  single -input,  single -output  system.  The  same  is  qualitatively 
true  of  the  parameter  |jQ||/ilR||  for  multidimensional  input  and  output 
( II A  II  5  norm  of  A)  . 

This  point  is  illustrated  by  the  simple  example  of  an  aircraft  pitch 
axis  control  system.  The  linearized  equations  assumed  to  represent  the  dy- 
Inamic  longitudiiiai-motiuns  of  the^airc raft  for  small- deviations  from  an-equi- 
librium  flight  path  are: 

^  i  •  L^oc  ^  Se 

where  ^  =  pitching  velocity 

ot  =  incremental  angle  of  attack 
Se  =  incremental  elevator  angle 

and  and  are  constants. 

[  dentrte-H  d  (  )  /  dtj. 


(7) 

(8) 
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The  elevator  is  driven  by  an  actuator  whose  dynamics  are  represented 
by  the  equation 

rSg  +  Sg.Sc  (9) 

where  =  command  input  to  the  actuator 

T  =  elevator  actuator  time  constant. 


These  equations  written  in  state  vector  form  are 


: — 
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R  =  [  1  ],  and  S  =  3x3  null  matrix 


(the  matrix  S  does  not  influence  the  solution  as  long  as  it  is  non- negative  def¬ 
inite  and  the  plant  is  completely  controllable  and  observable),  we  formulate 
the  performance  index 


2V 


lim 

T-fcOO 


+  +  Sc  )d7 


(11) 


Taking  numbers  representative  of  the  X-15  aircraft  at  a  Mach  number  of  4.  8 
and  an  altitude  of  77,  000  feet  (Table  1),  we  obtain  the  optimum  responses  to 
initial  angle  of  attack  shown  in  Figure  2  for  several  choices  of  Q  with  q,  , 

"  ^22- 

Some  of  the  responses  are  more  optimum"  than  others  on  the  basis 
of  what  we  know  to  be  a  desirable  transient  response,  although  all  the  closed- 
loop  responses  optimize  their  specified  performance  index.  The  choice  of 
relative  weighting  between  output  error  and  input  gives  a  wide  range  of  closed- 
loop  dynamics  and  there  is  no  a  priori  basis  at  present  for  choosing  the 
weighting  factor. 

However,  this  is  not  to  say  that  optimal  system  synthesis  is  useless 
in  flight  control  application.  Quite  to  the  contrary,  it  is  evident  from  the  ex¬ 
ample  that,  for  the  proper  choice  of  weighting  factor,  a  control  system  having 
an  acceptable  response  has  been  synthesized.  Figure  3,  showing;  the  control 
input  necessary  to  produce  the  responses  of  Figure  2,  indicates  that  quite 
reasonable  magnitudes  of  elevator  deflection  are  required. 

Figure  4  -  a  flow  diagram  of  the  optimal  system  -  shows  that  the 
system  would  not  be  difficult  to  mechanize.  Exaniination  shows  that  the  feed¬ 
back  gain  k3  (where  K  »  [k^k^k^])  serves  to  vary  the  equivalent  time  constant 
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of  the  elevator  actixator.  This  equivalent  time  constant  is  given  exactly  by 
the  relationship 

(4  )'  ,  ,  =6.67  (1  +  ki) 

closed- loop  ' 

Since  K  =  R’^G'P  from  equation  (5),  and  P  must  be  positive  definite,  then 
k3  =  1/2^  P33  >  0.  Therefore  for  this  application  the  equivalent  actuator 
closed- loop  time  constant  is  always  smaller  in  the  optimal  system. 

It  is  usually  desirable  to  reduce  the  number  of  feedback  parameters 
of  a  closed-loop  system.  This  fact  is  particularly  true  of  a  system  designed 
by  optimal  techniques  because  tlie  optimal  system  nominally  requires  that 
each  state  variable  be  fed  back  to  each  input  variable.  The  above  example 
shows  clearly  that  once  a  desired  optimal  response  is  achieved,  it  may  be  pos¬ 
sible  to  replace  an  existing  component  by  one  having  the  characteristics  re¬ 
quired  for  optimal  system  response. 

Another  interesting  point  can  be  made  from  the  example.  Comparing 
closed-loop  characteristic  equations  we  have: 


Q  Matrix 


Characteristic  Equation 


qil  =  q22  "  ®  (open  loop) 
qil  =  q22=.2 
^11  ^22  ^ 

‘^11  ""  "^22  ""  ^  ’  ^ 


X*  +  7.  13A*  +  20,  09  A  +  114.  3  =  0 
A*  +  10.  67A*  +  51,  7A  +  120.  5  =  0 
A’ +  12.  5  A*  +  72.  9A  +  129.  3  =0 
A*  +  14,  33A*  +  97.  3A  +  142.  1  =  0 


In  each  case,  as  Q  is  made  larger,  not  only  does  the  sum  of  the  roots  of  the 
characteristic  equation  become  larger,  but  the  sum  of  the  product  of  the 
roots  taken  two  at  a  time  and  the  product  of  all  three  roots  increases.  It  is 
suspected  that  the  characteristic  equations  for  other  optimal  systems  will 
also  exhibit  the  properties  shown  above. 


Throughout  this  study,  choice  of  a  performance  index  has  been  made 
using  the  two  steps  illustrated  in  the  example.  From  physical  intuition  we 
would  like  to  minimize  errors  in  oc  and  q.  So  we  include  of  and  q  in  the  per¬ 
formance  index,  Then  a  weighting  factor  is  selected  that  produces  a  desir¬ 
able  transient  response.  Thus  the  performance  index  is  used  as  a  perform¬ 
ance  index  -  that  is,  we  choose  elements  of  the  H  and  Q  matrices  to  minimize 
what  we  would  like  to  minimize  from  physical  considerations  —  and  it  is  used 
as  a  system  synthesis  tool  -  that  is,  the  weighting  factor  is  used  as  a  '*cut- 
ahd  try*  -  parameter.  The  real  criterion  of  performance  is  judgment  applied 
during  the  * 'cut-and-try*  *  procedure. 

The  beginnings  of  a  straightforward  synthesis  procedure  have  been 
made  by  recognizing  such  generalizations  as  the  image  root  interpretation  of 
the  single-input,  single -output  optimal  problem,  due  to  Chang,  It  is  possible 
that  more  general  results  can  be  obtained  by  examining  the  so-called  "reverse" 
optimal  splutipn.  That  is,  given  a  system  and  a  feedback  matrix,  what  per¬ 
formance  index,  if  any,  is  minimized  by  this  feedback  matrix.  The  equations 
of  the  *^reverse*^‘  solution  are  eqi^tions  (5)  and  (6)  with  P,  Q  (or  H*QH)  and 
R  as  unknowns  and  F,  G  and  K  as  knpvrns.  These  equations  can  be  written  as 


and 


(12) 

(13) 


F*P  +  PF  +  HQ*H  =  0 
RK  -  G*P  =  0 


Equations  (12)  and  (13)  are  now  linear  in  the  unknown  matrices.  By  virtue 
of  the  linearity  some  facts  are  readily  apparent.  Depending  on  the  number 
of  inputs  and  outputs  and  the  order  of  the  system  the  "reverse'*  solution  has 
morCf  less,  or  the  same  number  of  scalar  unknowns  as  equations.  When 
there  are  more  unknowns  than  equations,  a  solution  always  exists  for  any 
assumed  K  matrix  satisfying  the  inequaliUes  that  make  P,  Q  and  R  positive 
definite.  However,  if  the  number  of  vuiknowns  is  less  than  or  equal  to  the 
number  of  equations,  a  solution  exists  only  if  equalities  are  satisfied  by  the 
'  elements  of  the  K  matrix.  Both  the  inequalities  and  the  equalities  become 
important  when  the  designer  is  seeking  optimal  systems  that  are  physically 
realizable,  since  the  size  of  the  numbers  in  the  K  matrix  determines  di¬ 
rectly  the  feedback  configuration  by  determining  how  many,  if  any,  feedback 
gains  can  be  neglected. 


A  brief  example  of  the  equalities  that  arise  is  the  optimal  feedback 
for  the  plant  y(s)  /  u(s)  =  l/s^.  In  this  case  k^  =(k2)^/2  is  the  equality  and 
the  "reverse"  solutionis  q/ r  =  (kj^)^  . 


An  example  of  the  case  where  the  number  of  unknowns  exceeds  the 
equations  is  the  following: 


-4  1 

-2 
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and  K  =  [-.  828  -.038], 


then  the  "reverse"  solution  is  any  Q  and  R  satisfying 
and  R  =  [  r  ]>  r  >  0,  and  -.24  r  <  <lj^2  ^  l6.  12  r. 
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SPECIFIC  RESULTS 

1.  Lateral  Handling  Qualities  Problem 

The  criteria  for  the  design  of  the  primary  control  system  for  a  manned 
aircraft  are  generally  expressed  in  terms  of  what  are  known  as  handling 
qualities.  Handling  qualities  are  a  description  of  the  match  between  the  sta¬ 
tic  and  dynamic  characteristics  of  the  pilot  and  aircraft  to  accomplish  the 
mission  of  the  man-machine  combination.  Good  handling  qualities  are 
achieved  when  the  level  of  performance  of  the  combination  satisfies  the  mis¬ 
sion  objectives  with  the  least  amount  of  pilot  concentration  and  compensation. 

The  Cornell  Aeronautical  Laboratory  has  done  considerable  research 
in  the  past  fifteen  years  toward  determining  the  optimum  handling  qualities 
of  aircraft.^  (Reference  4  is  a  representative  recent  publicatipn. )  The  re¬ 
sult  of  this  research  has  led  to  a  definition  of  combinations  of  dynamic  and 
static  characteristics  judged  by  pilots  to  provide  good  handling  qualities. 
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From  these  characteristics#  and  from  general  past  experience,  a  mathemat¬ 
ical  model  of' the  desired  dynamics  of  an  aircraft,  can  be  constructed.  An 
optimum  control  system,  in  order  to  provide  good  handling  qualities,  must 
therefore  satisfy  two  criteria: 

1.  The  minimum  of  the  quadratic  performance  index 
must  be  obtained. 

2.  The  dynamic  characteristics  of  the  resulting  system 
must  be  within  the  area  defined  by  pilots  as  desirable. 

There  are  at  least  three  ways  this  problem  can  be  approached. 

The  first  is  to  obtain  by  trial  and  error  a  number  of  optimal  solutions 
From  these  solutions,  it  will  probably  be  possible  to  choose  a  combination  of 
H,  Q,  and  R  that  results  in  a  closed-loop  optimal  response  acceptable  from 
a  handling  qualities  point  of  view. 

The  second  approach  is  to  obtain  a  direct  relationship  between  the 
technique  of  design  by  performance  index  and  design  by  time  and  frequency 
domain  criteria.  Preliminary  steps  have  been  taken  along  this  path  via  the 
root- square  locus  and  the  '^reverse*'  solution  as  described  above. 

The  third  approach  is  to  include  the  mathematical  expression  for  the 
model  directly  in  the  formulation  of  the  system.  This  is  equivalent  to  re¬ 
defining  the  origin  of  the  output  space.  The  inclusion  of  the  model  redefines 
the  origin  to  be  a  desired  trajectory.  This  desired  trajectory  can  be  ex¬ 
pressed  explicitly  by  including  in  the  system  an  actual  physical  model,  or  it 
can  be  implied  by  expressing  the  performance  index  in  a  manner  that  'Causee 
that  part  of  the  performance  index  weighing  the  state  variables  to  go  to  zero 
as  the  optimal  response  approaches  a  desired  response. 

The  use  of  a  model  simplifies  the  design  procedure  considerably.  Its 
use  is  justified  for  a  number  of  reasons: 

1.  A  dual  performance  index  is  realizedi  one  pArt  expresses  dy¬ 
namic  criteria  on  the  output,  the  other  as  the  optimality  con¬ 
dition  that  minimizes  a  quadratic  function  of  control  action. 

The  best  use  is  made  of  the  coritrol  to  minimize  those  errors 
expressed  in  the  performaince  index. 

2.  To  approximate  the  respohse  of  the  model  more  and  more 
closely,  it  is  necessary  only  to  increase  the  ||Q||/j|R||  ratio. 

The  ||Q||/||R||  ratio  will  be  limited  by  physical  limitations 
on  the  feedback  gains. 

3.  The  time  constants  of  higher-order  plant  dynamics  can  be 
made  small  by  choosing  a  model  that  is  lower-order  than 
the  plant. 

The  model  is  included  in  the  performance  index  by  Kalman*  in  the 
following  way: 

The  model  is  expressed  as  the  matrix  L,  giving  the  autonomous  dynan;i 
system  ^  =  L  j  (f  a  ficticious  state  vector).  Forming  the  difference 


*  Reference  1,  Volume  II 


y  -  Ly  and  minimizing  this  difference»  the  plant  output  is  forced  to  act  like 
the  system  f  =  L  f  •  This  leads  to  the  performance  index 

T 

2V  =  S  [lly  -  Ly||*  +||u||*  ]  dT  (14) 

T-^00  t  w  IV 

where  the  notation||a  ||q  denotes  the  quadratic  form  at  Qa  where  Q  is  positive 
definite.  Substituting  M^uations  (1)  and  (2)  we  have 

lim  ^ 

2V  =  J  [|I(HF-LH)x +  HGu||Jj  +||u|||^  ]dr  (15) 


If  HG  =  0  (which  is  true  for  single -inputt  single -output  systems  with  the  num¬ 
ber  of  poles  exceeding  the  number  of  zeros  by  at  least  two),  then  we  define 
a  new  H  matrix  H  =  HF  -  LH  and  proceed  as-  before.  If  HG  ^  0,  new  La- 
grangian  and  Hamiltonian  functions  must  be  constructed,  but  the  solution  is 
similar  in  form  to  the  previous  one. 


The  optimal  control  system  for  achieving  good  lateral  handling  qual¬ 
ities  is  synthesized  as  follows,  using  the  model  formulation  proposed  by 
Kalman.  The  plant  is  assumed  to  be  an  X-15  -  type  aircraft  on  a  re-entry 
flight  path.  The  lateral-directional  motions  are  described  by 


fi  r  -  Yft  /B  +  A  y,  +  ty. 


(16) 


where  ^ 


sideslip  angle 
rolling  velocity 


IT  =  yawing  velocity 
=  rudder  deflection 
=  aileron  deflection 

and  the  remaining  parameters  are  constants  for  a  given  point  on  the  trajectory. 


The  aileron  and  rudder  actuators  are  described  by 


(17) 


IbB 


The  resulting  six- dimensional  system  is  written  in  state  vector  form  as 


Numbers  for  three  flight  conditions  are  listed  in  Table  2. 


(18) 


The  model  is  exactly  the  same  form  as  the  plant  with  actuator  dynam¬ 
ics  omitted  and  numbers  describing  a  near -optimum  aircraft  substituted  for 
the  re-entry  aircraft  numbers.  In  this  case  the  model  was  chosen  as  the 
characteristics  of  a  T-33  aircraft  at  25,  OOG  feet  and  250  knots  with  some 
artificial  damping  added  to  the  Dutch  roll  mode.  The  resulting  L  matrix  is 


0 

1 

0 

0 

-3.  72 

-2.  156 

.  0526 

.317 

^3 

-8.54 

0 

-2.  56 

2.  50 

_^4^ 

3.72 

2.  00 

0 

-.  317 

H 


^4  X  4 


0 

0 

0 

0 


0 

0 

0 

0 


From  previous  experience  we  know  th|it  matching  the/3  and  p  time  histories 
of  the  model  results  in  adeqxiate  overalll  matching.  Therefore  we  choose 
Q  and  R  to  be 


0  0  0  0 


0  0  0  0 


(Although  Q  is  non-negative  definite  here  for  convenience,  the  results  show 
no  detrimental  effects  of  this  choice.)  The  choice  of  Q  and  R  gives  the  per¬ 
formance  index 


2V  = 


ri«  /  ^  ‘133<P  -^3^)' 


(19) 


which  expresses  the  desire  to  make  the  aircraft  side  force  and  yawing  equa¬ 
tions  (combined  in  the  canonical  formulation)  and  rolling  equations  match  those 
of  the  model. 

Figure  5  shows  the  response  of  the  open-loop  system  and  the  closed- 
loop  optimal  system  at  the  first  flight  condition  listed  in  Table  2.  The  choice 
of  ^23  shown  in  the  figure.  The  aircraft  is  nearly  out  of  the  at¬ 

mosphere,  where  the  natural  frequency  of  the  Dutch  roll  mode  is  very  low, 
with  practically  no  damping.  The  optimal  response  does  not  follow  the  model 
closely,  but  achieves  good  Dutch  roll  damping  with  feedback  gain  values  that 
are  probably  physically  realizable  (with  the  possible  excv^ption  of  /3  feedback 
to  the  rudder  actuator). 

Figure  6  shows  the  same  series  of  responses  except  at  a  later  time 
during  the  re-entry.  Again  the  optimal  system  does  not  follow  the  model 
well  but  the  improvement  over  the  open -loop  is  considerable. 

At  a  still  later  time,  t  =  40  sec,  the  aircraft  follows  the  model  more 
closely  and,  from  a  handling  qualities  point  of  view,  would  probably  be  re¬ 
garded  as  a  good  aircraft.  The  transient  responses  are  shown  in  Figure  7. 

By  increasing  the  ratio  of  ||Q|[  to  IIRII.  it  is  presumed  possible  to 
make  the  aircraft  follow  the  model  as  closely  as  desired.  Figure  8  shows 
the  response  of  an  optimal  system  where  the  (|Q||  to  ]|R||  ratio  has  been  in¬ 
creased  by  a  factojr  of  5.  The  open-loop  aircraft  is  tnat  of  Figure  7.  The 
closed-loop  aircraft  follows  the  model  closely  and  it  rnay  be  presumed  that 
this  fidelity  can  be  improved  even  more  by  increasing  ||Q||/||R||*  Figure 
8  shows,  however,  that  as  this  ratio  is  increased,  a  corresponding  increase 
in  the  feedback  gains  is  required. 

If  one  desired  to  build  these  systems  \vithout  simplification,  two  ob¬ 
vious  disadvantages  of  the  optimal  systems  from  the  hardware  viewpoint  can 
be  seen  from  these  results.  Assuming  that  the  feedbacks  of  actuator  position 
can  be  incorporated  in  the  original  actuator  design,  one  must  still  provide 
sideslip,  roll  rate  and  yaw  rate  sensors.  Adding  a  differentiating  network 
for  the  sideslip  signal  brings  the  total  number  of  channels  up  to  eigkt.  This 
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is  a  lot  of  complexity.  Also,  the  feedback  gfiins  must  be  programmed  as  a 
function  of  time  or  changed  in  some  other  manner  during  the  re-entry.  This 
also  adds  complexity.  The  advantages  of  designing  a  control  system  of  this 
complexity  are  not  apparent  in  the  results. 

It  should  be  emphasized,  however,  that  these  results  do  not  preclude 
the  possibility  of  deriving  optimal  systems  that  are  less  complex  or  closely 
approximating  optimal  systems  with  less  complex  systems. 

2.  Booster  Pitch  Attitude  Control  Problem 

The  first- stage  attitude  control  system  of  a  large  booster  must  be 
designed  to  satisfy  several  criteria.  It  must  stabilize  a  usually  unstable 
airframe;  it  must  maintain  deviation  from  commanded  attitude  at  a  minimum; 
it  must  respond  to  wind  disturbances  so  that  angle  of  attack  does  not  exceed 
design  value,  at  the  same  time  limiting  drift  off  the  nominal  trajectory; 
and  it  must  accomplish  the  control  task  without  exceeding  engine  deflection 
limits  and  rate  limits.  The  quadratic  performance  index  by  itself  cannot  ex¬ 
press  all  these  desires.  However,  for  present  purposes  the  problem  is  re¬ 
stricted  to  that  of  maintaining  minimum  attitude  errors  —  a  simplification  that 
will  yield  results  and  yet  that  is  not  far  removed  from  the  real  problem. 

The  motions  of  the  flexible  booster  in  the  pitch  plane  and  the  engine - 
deflection  actuator  are  described  by: 

f  +  CO* ^  ©s,  k'S 

a,  s’*  a^6*a^S*S  ^  ( 20) 

where  ©|^  =  pitch  attitude  error  from  nominal  trajectory 
-  rigid  body  angle  of  attack 
S  =  engine  deflection 

=  local  attitude  of  structure  relative  to 
rigid  body  mode  (i^  mode) 

Sq  =  commanded  engiite  deflection, 
and  N,  ^  a^,  etc.  ar0  constants. 


Rigid  Body 


Structural 

Modes 


Actuator 


Written  in  state -  vector  form,  the  se  equations  are  as  follows 


(21) 

One  flight  condition  is  examined  here  -  Mach  Number  =  5  at 
160,  000  feet.  This  condition  is  typical  of  first- stage  burnout  (numbers 
given  in  Table  3). 

Since  we  desire  to  minimize  rigid  body  attitude  errors  but  have  no  in¬ 
terest  in  adding  to  the  structural  damping  or  changing  the  actuator  in  any 
way,  the  H  matrix  is  chosen  as; 

H=[1000000000  ]. 

This  makes  the  Q  and  R  matrices  both  scalars  and  leads  to  the  performance 
index 

T 

(The  choice  of  H  in  this  case  gives  a  plant  that  is  not  completely  observable. 
However  the  optimal  regulator  solution  may,  nevertheless,  be  unique,  due  to 
the  stability  of  the  unobservable  part  of  the  plant  -  the  structural  modes.) 

After  running  several  solutions,  the  ratio  q/r  =  10  was  chosen  as 
giving  a  good  compromise  between  speed  of  response  and  peak  engine  deflec¬ 
tions.  This  choice^  of  course,  had  to  be  based  on  previous  experience  with 
the  booster  problem.  The  transient  response  to  the  initial  condition  = 

1.0  deg,  =1.0  deg,  is  shown  in  Figure  9.  The  closed-loop  structural 

damping  is  no  greater  than  the  open-loop  damping  (f  =  .01  from  Table  3), 
but  ^is  is  to  be  expected  since  the  structural  mode  state  variables  are  not 
included  in  the  performance  index. 

The  feedback  gains  for  Figure  9  are 


172 


Sc  =  -Kx 


Sq  =  3.  16  +  1. 62  0,  +  .  018  oc  ^  -  .  246  S  -  .  0007  S 

-  .  194  X  10-4  g  .  (0)  -  (0)  ©s^  -  (0)  ©.^  -  (0)  .  (23) 

•  *« 

This  result  is  both  enco^uraging  and  discouraging,  for  although  oT^,  S  »  S, 

©5^  *  0,^  ,  ©Sjj  >  a^nd  ©5j  can  be  omitted  from  the  feedback  equation  without 
deviating  appreciably  from  the  optimum  system,  the  remaining  rigid-body 
variables  cannot  be  measured  without  sensing  some  structural  motion.  For 
sonae  sensor  positions,  this  structural  feedback  is  destabilizing. 

The  booster  representation  was  next  simplified  to  shed  some  light  on 
the  question  of  ignorable  higher -order  dynamics.  The  designer  would  like  to 
know  whether  it  is  legitimate  to  ignore  higher-order  dynamics  in  constructing 
an  optimal  solution  when  physical  intuition  indicates  that  they  can  be  ignored. 
For  the  flight  condition  examined  here,  one  would  suspect  that  the  structural 
modes  might  be  ignorable  (that  is,  ignorable  when  determining  the  feedback 
equation,  not  when  mechanizing  it).  This  was  shown  to  be  the  case,  because 
the  feedback  equation  for  the  booster  without  flexibility  is  the  same  as  equa¬ 
tion  (23)  with  ©5^  ,  ©5^  ,  etc.  omitted.  Dropping  the  actuator  and  the  heaving 
degree  of  freedom  from  the  booster  formulation,  we  find  the  feedback  equa¬ 
tion  is 


Se  =  3.  17  ©^  +  1.45  ©«  (24) 

Comparing  equations  (23)  and  (24),  we  suspect  that  the  omitted  dynamic  ef¬ 
fects  are  almost  but  not  quite  ignorable.  The  feedback  of  actuator  position 
is  the  only  significant  term  omitted  in  equation  (24). 


These  results  indicate  that  dynamic  effects  far  outside  the  bandwidth 
of  interest  can  be  ignored  in  determining  the  optimum  feedback  configuration 
much  the  same  as  they  are  ignored  in  conventional  design  practice. 


The  final  result  in  the  booster  application  is  the  comparison  of  the 
optimal  system  with  a  realizable  system.  Figure  10  is  the  block  diagram  of 
a  realizable  system.  This  loop,  taken  from  a  previous  study,  was  not  de¬ 
signed  to  approximate  the  feedback  of  equation  (23)  except  in  the  sense  that 
low-pass  filters  are  included  to  greatly  attenuate  the  feedback  of  ©5  and 
The  compensation  is  lead- la g^  However,  plotting  2V  from  equation '(22)  ^ 


2V 


deg^ 

sec 
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The  realizable  systenif  in  this  flight  condition,  approaches  the  optimal  sys¬ 
tem  quite  closely  although  the  transient  response  is  slightly  oscillatory.  As 
gain  is  increased  from  the  minimum  point,  system  oscillations  increase 

(so  I  increases),  but  the  bandwidth  also  increases  so  that  f 

o'  • 

continues  to  decrease.  Figure  11  shows  the  transient  response  at  a  gam  of 

2.  2.  As  is  shown  on  the  plot,  the  error  portion  of  the  performance  index  is 
almost  equal  to  that  of  the  optimum,  but  the  control  input  integral  is  consid¬ 
erably  larger. 


CONCLUSIONS 

Only  some  of  the  results  of  the  study  mentioned  in  the  introduction  have 
been  included  in  this  paper,  but  they  illustrate  the  general  experience  gained. 
Both  success  and  failure  have  been  encountered  in  choosing  performance  in¬ 
dices  giving  satisfactory  optimum  systems.  However,  the  following  conclu¬ 
sions  can  be  made: 

1.  State  variables  characterizing  higher -order  system  dynamics 

.  need  not,  in  some  cases,  be  fed  back  to  obtain  an  optimal  system 
if  they  do  not  appear  explicitly  in  the  performance  index. 

2.  Choosing  the  performance  index  from  physical  intuition,  ex¬ 
cept  for  the  relative  weights  given  to  the  error  and  the  con¬ 
trol,  works  fairly  well  in  obtaining  satisfactory  transient 
response,  although  this  procedure  will  not  usually  lead  to  a 
feedback  configuration  that  is  easily  realized  physically. 

3.  Further  work  is  needed  to  establish  general  relations  between 
the  performance  index  and  the  resulting  feedback  configuration 
if  the  optimal  control  theory  is  to  be  useful  as  a  practical 
system  synthesis  tool. 
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TABLE  1 

X*15  -  AIRCRAFT  LONGITUDINAL  DERIVATIVES 
M  =  4.  8,  h  =  77,  000  feet 


M  =  -.  132  sec 

M  ^  =  -.  046  sec 

=  - 17.  1  sec 

Me  =  - 12.  2  sec 

®e 


L  =  .  277  sec 

Lj  =  .037- sec 

T  =  .  15  sec 


TABLE  2 

RE-ENTRY  AIRCRAFT  DERIVATIVES 


time  after 
re-entry 
initiation 
(sec) 

Mach 

No. 

h 

(1000 

ft) 

-1 

sec 

L./3 

-2 

sec 

L 

P 

-1 

sec 

L 

r 

-1 

sec 

-2 

sec 

0 

5.  3 

.0208 

mm 

.492 

20 

5.6 

.0637 

mg 

.932 

40 

5.5 

147 

.5140 

SMi 

+.  0376 

2.95 

N 

r 

L. 

•» 

*«. 

g/v 

-I 

sec 

-2 

sec 

-2 

sec 

-2 

sec 

see 

-2 

sec 

-1 

sec 

-.0015 

-.  335 

.  150 

-.0161 

-.  178 

.00552 

-.  0028 

-.  647 

.  353 

-.0311 

-.  349 

.00550 

-.0103 

-5.  11 

-2.  32 

-.249 

-1.43 

.00535 

TABLE  3 


BOOSTER  DERIVATIVES 
Mach  No,  =  5^  0  h  =  I60,  000  feet 


036  (sec" 

= 

56.  3  X  10’^  (sec) 

3.02  (sec’^) 

= 

.  809  X  10“^  (sec)^ 

.00175 

^3  = 

. 00447  X  10'^  (sec) 

.0292  (sec“^ 

= 

15.  1  (rad/sec) 

.  0148  (sec" 

= 

86.  1  (rad/  sec) 

.  0012  (sec" 

S’  = 

.01  (--.) 

.  936  X  10"^  (-.-) 

= 

6.  21  (sec'^) 

ky  = 

.  00515  (---) 

^2  ^ 

59.  6  (sec*^) 

k-  = 

.0843  (---) 

3 
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Figure  1.  Optimal  Seoond-Order  Systems 
(Single  Input,  Single  Output) 


Figure  3.  Control  Input  (  8  )  Response  to  Initial  Angle  of  Attack 
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Figure  7,  Optimal  Regulator  Response  Simplified  X-15 
Lateral  Dynamics  Mach  =  5. 5  h  =  147,000  Ft 
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Optimal  Regulator  Response  Simplified  X-15 
lateral  Dynamics  Mach  f  5. 5  h  s  147,000  Ft 


Sr  and  05, 


Figure  9.  Optimum  Regulator  Response  to  Initial  9  and  a„ 
Booster  at  Mach  5,  160,000  Ft 
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TIME-SEC 


- HIGH-GAIN  SYSTEM 

00  OOg 

2V  =  J  10  Gr  dt  *J df  =  4.12  ♦  12.28 
°  o  .  16  40  dEG^-SEC 


-—OPTIMAL  SYSTEM 

2Vmin  =/  10  ®R  <!♦  *1%^^ 

°  °  » 5.51  DEG -SEC 


Figure  11.  Comparison  of  Optimal  Regulator  With  a  Realizable  Booster 
Control  System  (Gain  =  2,2) 
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ABSTRACT 


A  three-dimensional  trajectory  optimization  coni- 
puter  program  is  described  for  determining  vehicle 
performance  boundaries.  Simultaneous  programs  for 
angle -of -attack  and  bank-angle,  with  specified  initial 
and  terminal  conditions,  are  determined  by  this  pro¬ 
gram  using  steepest-ascent  computational  procedures. 
Terminal  constraints  in  the  form  of  upper  bounds  on 
total  stagnation  point  heat  absorption  and  total  accel¬ 
eration  dose  and  in-flight  constraints  in  the  form  of 
upper  bounds  on  altitude  and  stagnation  point  heating 
rate  are  also  included.  The  vehicle  is  treated  as  a 
mass  particle  and  the  earth  is  approximated  as  on 
oblate-spheroid  with  an  ARDC  atmosphere  rotating 
with  the  earth.  Only  coordinated  turn  maneuvers  are 
considered.  The  terminal  functions  which  can  be 
optimized  include  all  the  state  variables  individually 
and  a  number  of  functions  of  the  state  variables.  These 
computer  programs  are  specifically  oriented  for  com¬ 
putations  on  the  IBM  7090. 

A  numerical  example  is  given  for  a  SORTIE  con¬ 
figuration  consisting  of  a  three- stage  booster  system 
and  a  payload,  the  re-entry  vehicle,  SORTIE.  Three 
separate  but  related  performance  boundaries  were 
computed  for  this  configuration:  I)  the  boost  boundary 
for  the  three-stage  booster  system,  2)  SORTIE  re-entry 
and  glide,  3)  SORTIE  lateral  maneuvers  in  the  recovery 
area. 
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1.  INTRODUCTION 

The  determination  of  the  performance  capabilities  of  a  hypersonic  vehicle 
requires  a  three-dimensional  analysis  because  of  the  rotation  of  the  atmosphere 
with  the  earth,  the  oblateness  of  the  earth,  and  the  fact  that  the  vehicle  is  capa¬ 
ble  of  making  lateral  turns.  Constraints  on  acceleration  dosage,  heating, 
heating  rate,  and  altitude  may  also  be  placed  upon  the  vehicle. 

To  map  out  the  performance  boundaries  of  a  given  vehicle  by  a  cut-and-try 
approach  requires  many  individual  tries.  The  technique  presented  in  this 
report  allows  one  to  proceed  in  a  systematic  and  efficient  manner  to  map  out  the 
performance  boundaries.  It  requires  the  use  of  a  high-speed  computer  to 
determine  each  trajectory  and  employs  a  steepest-ascent  or  successive - 
improvement  method. 

Two  major  computer  programs  are  presented:  1)  for  a  gliding  vehicle  and 
Z)  for  a  vehicle  which  has  a  fixed  direction  of  thrust  relative  to  the  vehicle. 

Only  coordinated  turns  are  considered  for  both  gliding  and  thrusting  vehicles; 
thus  angle -of -attack  and  bank  angle  are  the  control -variables.  The  vehicle  is 
approximated  by  a  mass  particle  acted  upon  by  aerodynamic,  propulsive,  and 
gravitational  forces.  This  represents  a  considerable  simplification  over 
treating  the  vehicle  as  a  rigid  body,  which  would  involve,  in  addition,  the  pitch, 


This  study  program  was  supported  in  part  by  funds  under  USAF  Contract 
No.  AF  33(61 6):-8300,  Task  No.  143107  of  Project  1431,  This  paper  consists 
of  a  condensation  of  References  (10-12). 
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yaw  and  roll  dynamics «  It  is  believed  that  this  approximation  is  reasonable 
when  determining  performance  boundaries* 

In-flight  constraints  are  satisfied  by  the  use  of  penalty  functions  which 
effectively  converts  them  into  terminal  constraints.  Two  in-flight  conetraints 
are  considered,  either  one  of  which  may  be  applied  in  a  given  problem; 

1)  An  upper  bound  on  altitude  during  the  flight  and  2)  an  upper  bound  on  the 
heating  rate  at  the  stagnation  point  during  the  flight. 
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2.  COORDINATE  SYSTEMS 


Z,  1  Spherical  Coordinate  System 

The  choice  of  the  coordinate  systems  is  an  important  part  of  the  optimiza¬ 
tion  analysis.  Since  the  atmosphere  arid  the  desired  landing  point  both  rotate 
with  the  earth,  it  seems  that  a  simple  and  desirable  coordinate  system  to  use 
is  the  conventional  geographic  coordinate  system.  This  coordinate  system  is  a 
spherical  polar  coordinate  system  described  by  a  radius  from  the  center  of  the 
earth,  r,  a  co-latitude  angle  measured  from  the  North  Pole,  6,  and  a  longitude 
angle  measured  from  the  Greenwich  dr  prime  meridian,  (See  Figure  1). 

The  earth  is  assumed  to  be  an  oblate  spheroid  with  an  equatorial  radius,  R  , 
of  20.925631  x  10  feet  and  a  polar  radius,  R^,  of  20.855965  x  10°  feet. 

(See  Reference  1.  )  Thus  the  radius  of  the  earth  becomes  a  function  of  the  co¬ 
latitude  and  can  be  given  as 


Since  the  second  term  in  the  radical  is  much  less  than  unity,  the  radical  can 
be  expanded  into  a  series.  Retaining  only  the  first  two  terms,  the  radius  of  the 
earth  is  given  approximately  by: 


R  -  R 

R(e)  =  %  (1-  ■  -  -P’  cos^  e) 

e 


(2-2) 


Because  the  earth  is  an  oblate  spheroid  of  eccentricity  near  unity,  the  radius 
vector  is  not  quite  perpendicular  to  the  local  earth  mean-surface.  However, 
the  maximum  angle  between  the  radius  vector  and  the  normal  to  the  spheroid 
is  11.  5  minutes  of  arc  and  occurs  at  a  co-latitude  of  45*.  Thus  the  error  in 
assuming  that  the  altitude  is  given  by: 
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Figure  1  -  Geographic  Position  Coordinates  r,  ^ 


h  =  r  -  R(0)  (2-3) 

is  negligibly  small.  The  atmospheric  density  is  assumed  to  be  a  function  of 
altitude  h  above  the  earth's  surface  and  is  taken  to  be  the  standard  1959  ARDC 
atmosphere  model. 


Auxiliary  Coordinate  Systems  6 


Two  auxiliary  coordinate  systems  are  used  for  convenience  in  determining 
vehicle  performance.  One  coordinate  system  is  described  by  the  angles  0,0. 
This  system  places  a  "north  pole"  at  the  initial  point  of  the  problem  located  by 
the  geographic  angles  6^  and  The  "prime  meridian"  is  oriented  to  lie  in  the 
great  circle  plane  of  the  initial  flight  path  direction.  Thus,  6  is  the  great 
circle  distance  from  the  initial  point  and  <jt  is  the  lateral  longitude  from  the 
initial  great  circle.  (See  Figure  2). 

The  "equator"  of  the  second  coordinate  system  is  located  on  the  initial 
great  circle  and  the  "prime"  meridian  lies  through  the  initial  point  at  right 
angles  to  the  "equator",  p  is  the  longitude  measured  along  the  "equator"  and 
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h  =  r  -  R(e)  (2-3) 

is  negligibly  small.  The  atmospheric  density  is  assumed  to  be  a  function  of 
altitude  h  above  the  earth's  surface  and  is  taken  to  be  the  standard  1959  ARDC 
atmosphere  model. 

Tfjt  >!< 

Z,  Z  Auxiliary  Coordinate  Systems  Q  ,  ^  and  p;,  v 

Two  auxiliary  coordinate  systems  are  used  for  convenience  in  deterniining 
vehicle  performance.  One  coordinate  system  is  described  by  the  angles  d  ,  ^  . 
This  system  places  a  "north  pole"  at  the  initial  point  of  the  problem  located  by 
the  geographic  angles  d  and  ^  .  The  "prime  meridian"  is  oriented  to  lie  in  the 
great  circle  plane  of  the  initial  flight  path  direction.  Thus,  6  is  the  great 
circle  distance  from  the  initial  point  and  0  is  the  lateral  longitude  from  the 
initial  great  circle.  (See  Figure  2). 

The  "equator"  of  the  second  coordinate  system  is  located  on  the  initial 
great  circle  and  the  "prime"  meridian  lies  through  the  initial  point  at  right 
angles  to  the  "equator".  |ji  is  the  longitude  measured  along  the  "equator"  and 
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Figure  2  -  Auxiliary  Position  Coordinates  0*  or  v 


u  is  the  great  circle  distance  from  the  initial  great  circle  plane,  i.  e.  latitude 
from  the  ’’equator".  (See  Figure  2). 

The  radius  vector,  is  the  same  in  all  three  coordinate  systems. 
Knowing  the  initial  co- latitude,  6^,  the  initial  longitude,  (f>^,  and  the  initial 
flight  path  heading  angle,  3^,  along  with  the  present  co-latitude,  6,  and  longi¬ 
tude,  the  coordinates  0  ,  ,  p  and  u  can  be  found  from: 


cos  V  cos  u  s  cos  0  =  sin  0  sin  0  cos  (A  -  4>)  +  cos  0  cos  0 

O  '  Tq  Q 

sjc  >ic 

sin  V  s  sin  0  sin  6  s  sin  0  cos  0  sin  B 


-  Sin 


in  0  ^  sin  3^  cos  (^^  -  +  cos  3^  sin  j  (2-4) 

^  cos  3q  cos  0^  cos  (4^  -  4») 

sin  3  sin  (^  *  4>)  I  -  cos  B  sin  0  cos  0 
no  '^o  J  *^0  0 


*  * 

cos  V  sin  ^  s  sin  0  cos  ^  s  sin  0 
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and 

♦ 

tan  ■  tan  6  cos  ^ 
•in  V  ■  sin  6  sin  ^ 


(2-5) 

(2-6) 
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3.  STATE  EQUATIONS 


3  1  Equations  of  Motion 

Vehicle  position  is  described  by  the  spherical  coordinates  r,  6,  and^. 

A  ^  A 

(See  Figure  3).  Let  r,  G,  and  0  be  unit  vectors  in  the  direction  of  increasing 
r,  0  and^,  respectively;  then  the  velocity  of  the  vehicle  relative  to  the 
rotating  atmosphere  is  given  by 


A  A  A 

V-sur  +  ve  +  w<j) 


(3-1) 


It  is  convenient  at  times  to  describe  the  velocity  vector  by  its  magnitude  V,p, 
its  elevation  angle  above  horizontal,  y ,  (actually  above  an  r  »  constant  surface), 
and  its  azimuth  angle  (or  heading)  counter  clockwise  from  south,  p.  Thus, 
from  Figure  4, 


2'  2  ^  2 
+  V  +  w 


tan  0  = 


tan  y  s 


y 


2  u.  2 

V  +  w 


'h  *  yy* 


w 


u 


H 


(3-2) 


The  relationship  between  position  and  velocity  components  is 


r 

e 

♦ 


V 

r 


(3-3) 


w 

r  sin  6 
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Figure  3  -  The  Coordinate  System  and  Forces  of  the  Problem 
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are  given  by 


s  -m 


^  (1-3  co»^ 


(»|]  +  A^  +  T^ 


mZfi  JR, 

j - -  cos  e  sin  $  +  Ag  + 


(3-6) 


The  thrust  components  T^,  Tg  and  T^are  given  in  terms  of  the  components 
of  thrust  along  the  vehicle  aixes  by  the  transformation: 


1 

u 

H 

1 _ - 

- 1 

*12 

^13 

r~ - 

X 

H 

1 _ _ _ 

'  s 

^21 

*22 

*23 

T 

y 

^31 

*32 

c»> 

T 

z 

The  functions  of  p,  y,  a,  and  a,  given  in  the  Appendix.  Two  constant 

offset  angles  (i.p,  5  j)  orient  the  thrust  vector  relative  to  the  vehicle  axis  system. 
The  thrust  components  in  the  vehicle  axis  system  are  given  as 

T^  *s  T  cos  i.j.  cos  6.J. 

Ty  a  T  cos  sin  6.J.  (3-8) 

T_  ■  T  sin  i_ 

Z  X 


where  T  ie  a  knpwn  function  of  time  and  stage  number. 

The  aerodynamic  force  components  Aq,  and  A^are  given  in  terms  of 
force  components  perpendicular  and  parallel  to  the  velocity  vector  (lift  and  drag) 
by  the  transformation: 
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A 

r 

\i 

^2" 

l" 

m 

^21 

^22 

_-D_ 

^31 

^*32 

(3-9) 


The  s  functions  of  p,  y ,  and  a,  given  in  the  Appendix. 

Derivations  of  the  thrust  transformation  matrix^  and  the  aerodynamic 
force  transformation  matrix  are  given  in  the  Appendix. 


3.  2  In"»Flight  Constraints  on  Altitude  and  Heating  Rate 

Two  in-flight  constraints  are  considered,  either  one  of  which  may  be  used 
in  any  given  problem: 


1)  An  upper  bound  on  the  altitude,  h,  during  the  flight. 

2)  An  upper  bound  on  the  heating  rate  at  the  stagnation  point,  q, 
during  the  flight. 


These  constraints  arc  handled  by  the  use  of  penalty  functions: 

2 


P  = 


P  = 


h-A 


F  =  A  , 
p  pi 


"p? 


PI 


/q'^2 

^2 


h  >  A 


p2 

h  <  A  , 

p2 

q  >  Ap2 


q  <  A 


P2 


(3-10) 


(3-11) 


where 

q  =  p 


1/2 


See  Re£erence(2)  for  details  about  penalty  functions. 
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If  the  nominal  trajectory  exceeds  the,  desired  upper  bound  on  altitude  (say)  then 
the  terminal  value  of  p  will  be  positive.  The  successive -improvement  scheme 
will  reduce  this  terminal  valv.e  of  p  to  a  small  positive  quantity  which  will  cause 
the  final  trajectory  to  exceed  the  desired  upper  bound  on  altitude  by  only  a  negli¬ 
gibly  small  amount. 

3.  3  Terminal  Constraints  on  Pilot  Acceleration  Dose  and  Total  Heat  Absorbed 

A  measure  of  the  acceleration  dose  absorbed  by  a  human  pilot  subjected  to 
an  acceleration  history  a(t)  is  given  by 


where  rfa)  is  the  time  (empirically  measured)  that  a  human  pilot  can  stay  use¬ 
fully  conscious  under  a  constant  acceleration  level  a.  Whenever  e  exceeds  a 
value  of  unity,  it  is  assumed  that  the  pilot  has  had  an  excessive  dose  of  acceler¬ 
ation  and  cannot  any  longer  perform  a  useful  piloting  operation.  Figure  5 
shows  a  plot  of  T  vs  a  using  data  from  Reference  (3). 

An  approximate  measure  of  the  rate  of  heat  absorption  at  the  stagnation 
point  of  an  ablative  nose  is  given  by 


A 

q 


pi/ 2  V 


3 


T 


(3-13) 


where  A.  is  a  constant.  This  relation  does  not  include  the  heat  due  to  radiation 

q 

from  the  hot  gases  surrounding  the  nose.  It  should  be  noted  that  A^  can  be 
specified  so  that  q  will  be  a  normalized  value.  For  further  details  on  the  heat 
absorption  equation  see,  for  example,  Reference  (4). 
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Figure  5  -  Limiting  Acceleration  Dose  for  Useful  Consciousness «  r  vs  a 
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3,  4  Summary  of  State  Equations 


are 


The  eight  first-order  non-linear  differential  equations  of  state 
summarized  below: 


u  s  —5?  +  r  Q  ^  sin^  0  +  2  fl  w  sin 
re  e 


+  cos  a  Cj^  V 


H  J  7  -T— 


3  cos^  6)  +  — —  (3-14) 

'  m 


B  -  HI  +  ^  +  r  n  ^  sin  0  cos  0  +  2  a  w  cos  0 

r  r  e  e 


-  eJ  f  c 

2m  L  D 


u  V  V_ 

X 

V  +  cos  O'  C-  - ■■■  +  sin  o’  C 

T  ^ 


\v  V 


^  JRf 


+  Z  — — -pi —  cos  0  sin  0  +  — — 
4  m 


(3-1^) 


.  UW  V  W  Ctn  6  -y  t  •  O  O  \ 

w  =  -  - - 2  fl  (u  sin  0  +  V  Gos  0 ) 

r  r  e 


p  s  r  u  w 

P—I^CjjwV^  +  cosoCl  - sin 


V  T  T  T  . 

a  C,  I  +  -±.  (3-16) 


r  =  u 


0  =  - 


(3-17) 

(3-18) 


♦  = 


r  sin  0 


e  =  F 


=  F 


(3-  19) 
(3-20) 
(3-21) 
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where 


m  -  vehicle  mass 

^e  ■  *o  *^e^  *  1.407698  X  10 sec"^ 

■  20.925631  X  10^  ft 

J  »  1623.41 X  10'^ 

fig  =  7.  29211508  X  10"^  rad/sec 

L  ■  Cj^  (a«  M)  — 2  ® 

P  V  ^ 

D  ■  (or,  M)  - 2"'  S  m  drag 

p  «  p  (h)  s  air  density  ARDC  tables 
S  s  reference  area 
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4.  OPTIMUM  PROGRAMMING  PROBLEMS  AND 
THEIR  SOLUTIONS 


4. 1  Optimum  Programming  Problems 

A  process  which  develops  in  space  and/or  time  in  which  one  or  more 
control  variables  must  be  programmed  to  optimize  a  performance  criterion  is 
an  optimum  programming  problem.  The  problem  is  to  determine,  out  of  all 
possible  programs  for  the  control  variables,  the  one  program  which  maximizes 
(or  minimizes)  the  performance  criterion  (often  one  of  the  thermal  quantities) 
while  simultaneously  yielding  specified  values  of  certain  terminsd  quantities 
and  satisfying  the  in-flight  constraints  of  the  system. 

A  formulation  of  a  general  class  of  optimum  programming  problems  is 
given  below: 

Let  the  system  or  process  be  described  by  n  first-order  non-linear  (or 
linear)  ordinary  differential  equations  of  the  form 


dx. 

1 


=  X. 


■“f*  {^1’  .  •>  X  $  Q!|  f  CK  -y  9  .  . 


a  ,  t) 
m  ' 


(4-1) 


where  i  *  1,  2,  3,  . ,  . ,  n,  and  the  control  variables  are  given  by  ay 

j  =  1,  2,  3,  . . . ,  m,  which  are  free  to  be  selected.  The  problem 
is  to  determine  the  a.  *s  in  the  interval  <  t  <  T,  such  that  some  qu^tity 


♦  =  ♦  [  x.(T),  T  ] 


(4-2) 
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is  a  maximum  (or  a  minimum),  subject  to  satisfying  the  physical  equations 
given  by  Equation  (4.  1)  and  certain  other  specified  terminal  constraints 


{x.(T),  T) 
ip  {x.(T),  T) 


(4-3) 


«  a  (  X.  (T),  T  )  a  0  (4-4) 

All  the  terminal  constraint  functions  are  known  functions  of  the  state  variables 
and  time.  The  stopping  condition,  12,  determines  the  final  time  T  and  is  a  known 
function  of  x^  (T)  and  T. 


4,  2  Classical  Approach  to  Solving  Optimum  Programming  Problems 

The  classical  approach  to  solving  optimum  programming  problems  is 
by  the  use  of  the  calculus  of  variations.  In  the  classical  formulation,  these 
problems  are  two-point  boundary  value  problems  for  a  set  of  non-linear 
ordinary  differential  equations.  The  boundary  conditions  for  the  physical 
equations  are  usually  specified  at  the  initial  point,  and  the  boundary  conditions 
for  the  Euler- Lagrange  equations  are  usually  specified  at  the  terminal  point. 
Thus,  solving  the  set  of  equations  numerically  requires  that  a  *’guess"  be 
made  for  the  initial  values  of  the  Lagrange  rnultipliers  in  the  Eule r- Lagrange 
equations  and  the  two  sets  of  equations  must  then  be  integrated  to  the  terminal 
point.  A  check  of  the  terminal  boundary  conditions  is  then  made  to  see  how 
badly  they  were  missed.  A  correction  niust  then  be  made  to  the  ’’guessed" 
initial  values  to  improve  agreement  with  the  desired  terminal  boundary 
conditions.  This  process  must  be  repeated  until  all  of  the  terminal  boundary 
conditions  are  satisfied.  This  process  is  hot  only  tedious,  expensive  and 
frustrating,  but  great  patience  must  be  exercised  before  any  worthwhile 
results  can  be  expected. 
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3  Steepest- As  cent  Technique  for  Solving  Optimum  Programming  Problems 

A  systematic  and  efficient  method  of  solving  an  optimum  programming 
problem  on  a  high-speed  digital  computer  using  the  steepest- ascent  technique 
will  be  described. 

Recently,  a  little- known  procedure  for  determining  numerical  solutions 
to  complicated  optimum  programming  problems  has  been  revived  by  Kelley 
(Reference  2)  and  Bryson  et  al  (References  6,  7  and  8).  These  references 
present  a  number  of  examples  using  the  steepest- ascent  technique. 

In  this  method  the  control  functions  o  j(t)  are  guessed  and  a  trajectory  is 
computed.  The  equations  adjoint  to  the  equations  describing  small  perturbations 
about  this  trajectory  are  integrated  backwards  over  the  trajectory  with  suit¬ 
able  boundary  conditions.  The  results  of  this  integration  yield  the  impulse- 
response  functions  of  the  final  conditions  to  small  changes  in  the  programs. 

By  suitably  combining  these  impulse- response  functions,  changes  in  the  Oj(t) 
programs  can  be  made  which  will  simultaneously  increase  the  final  value  of  the 
optimizing  function  and  approach  more  closely  the  desired  values  of  the  terminal 
constraints.  This  process  is  repeated  until  it  becomes  apparent  that  only 
small  changes  are  occurring  in  the  optimizing  function  and  for  all  practical 
engineering  purposes  this  last  trajectory  is  the  optimum  trajectory. 


4.  4  Adjoint  Equations  for  Steepest- Ascent  Optimization 
The  influence  (adjoint)  functions  are  devoted  by 


X  ,  \  , 

V  w 


^  ,  X-,  X.,  X  ,  \  (4-5) 

u'  V'  w'  r  0  <p  e  q 

where  the  subscript  refer  to  the  physical  variable  associated  with  that  particular 

influence  function.  The  eight  first-order  non-linear  differential  equations  of 

state  given  in  Section  3.  4  can  be  written  in  the  shorthand  form  of  Equation  (4-1) 

where 


(V  V  ^3*  “2 . ’“m) 


(u,  V,  w,  r,  6,  <t),  e,  q,  a,  a)  (4-6) 


The  adjoint  differential  equations  can  be  written  as: 


X. 

1 


i  =  1 ,  2,  3,  ,  .  .  ,  8 


(4-7) 
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where  (A^.  . A-g)  =  (A^.  X^,  X^,  X^) 

And  the  impulse  response  functions  for  the  angle  of  attack  and  the  bank  angle, 
a  and  a,  respectively,  are 


The  ten  relationships  given  by  Equations  (4-7)  and  (4-8)  are  quite  lengthy 
when  written  out  but  can  be  obtained  by  straight  forward  differentiation.  Hence, 
the  ten  relationships  are  not  given  here,  however,  they  are  written  out  in  full 
Reference  (10),  which  also  gives  details  on  the  boundary  conditions  and  the 
computational  procedures  used,  as  well  as  other  details  on  the  computer 
programs. 
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5.  EXAMPLE  PROBLEM-THE  PERFORMANCE  BOUNDARIES 
FOR  A  SORTIE  CONFIGURATION 


5. 1  The  Example  Problem 

As  an  example  three  separate  but  related  performance  boundaries  for  a 
typical  three- stage  booster  system  with  a  re-entry  vehicle  as  its  payload  is 
presented.  This  example  demonstrates  the  main  features  of  computer  programs 
developed  to  solve  trajectory  optimization  problems.  The  first  part  demonstrates 
the  practical  way  of  determining  control  programs  for  a  booster  system  with 
specified  terminal  conditions.  The  second  part  of  the  example  presents  an 
example  problem  with  an  in-flight  constraint  during  the  re-entry  phase  for 
near  escape  speeds.  Finally,  the  third  part  presents  the  computer  program 
capability  to  determines  lateral  maneuvers  for  a  gliding  vehicle.  Each  of  these 
examples  are  presented  in  turn. 
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6.  THE  VEHICLE  STAGE  AND  AERODYNAMIC  DATA 


The  booster  used  in  calculating  the  SORTIE  performance  was  the  Atlas - 
Agena.  Figure  6  shows  the  lift  and  drag  characteristics  for  the  vehicle  during 
first  stage  and  second  stage  boost.  Figure  7  shows  the  lift  and  drag  character¬ 
istics  for  the  third  stage  boost  and  for  the  re-entry  SORTIE  vehicle.  Figure  8 
shows  a  sketch  of  the  SORTIE  vehicle  and  Figure  9  presents  the  L/D  ratio  vs 

a  for  this  vehicle.  (L/D)  is  equal  to  1.  59  at  a  =  17.  5®. 

'  '  'max  ^ 


211 


J  — 1 - f - 1 

2  4  6  t  to 

a  ANCLC  OF  ATTACK 


Figure  8  -  SORTIE  Vehicle 


7.  BOOST  TRAJECTORY 


The  trajectory  optimization  computer  program  for  vehicles  with  thrust  has 
been  demonstrated  by  determining  the  maximum  velocity  at  burnout  with  speci-* 
fifed  terminal  values  for  the  altitude  and  flight-path  angle  for  the  three- 
stage  booster  system.  The  specified  terminal  conditions  were  h^  = 

400,  000  ft  and  y  £  =  -5  degrees.  The  initial  conditions  for  the  optimization 
procedure  were  determined  by  integrating  the  equations  of  motion,  neglecting 
the  effects  of  drag,  for  a  period  of  15  seconds  for  a  vertically  launched  vehicle. 
The  following  conditions  were  used  as  the  initial  conditions  for  the  optimization 
of  the  launch  trajectory: 

=  190  ft /sec 

Vo  =  89* 

h  =  1300  ft 

o 

i3  =  70® 

o 

d  =  61.5® 

o 

6  =  -80.  5* 

o 

The  angle -of- attack  program  for  the  nominal  trajectory  was  taken  from  a 
preliminary  series  of  boost  trajectories  which  had  been  run  to  check  out  the 
computer  program.  It  should  be  noted  that  the  terminal  values  of  h  and  y  for 
the  nominal  case  were  at  least  one  order  of  magnitude  too  large.  The  nominal 
trajectory  had  the  following  terminal  conditions: 

=  28,530  ft/sec 
hj  =  4,  603,  350  ft 
Vf  =  32,39* 
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The  computer  program  in  34  iterations,  very  effectively!  reduced  the  terminal 
values  of  h  and  y  to  within  90%  of  the  specified  values.  The  terminal  conditions  at 
the  end  of  34  iterations  were 

=  35,  620  ft/sec 
=  430,  579  ft 
=  -4.96* 

with  an  increase  of  7090  ft/sec  in  the  terminal  velocity,  nearly  a  25%  improve¬ 
ment  over  the  nominal  case.  The  altitude  differential  has  been  reduced  from 
4,  203,  350  ft  to  430,  579  ft  and  the  flight- path  angle  is  within  .04  degree  of  the 
specified  value.  Two  additional  series  of  iterations  were  made  and  the  results 
of  these  runs  are  presented  in  Table  1. 


TABLE  I 

SUMMARY  OF  SUCCESSIVE  IMPROVEMENTS  FOR  THREE-STAGE 

BOOST  TRAJECTORY 


Case 

V,j.^-ft/sec^ 

Yf-deg 

h^-ft 

o 

Nominal 

28, 530 

32.  391 

4,603,350 

1.0  X  10"^ 

34 

35,620 

-4. 963 

430,579 

1.0  X  10"^ 

404, 696 

dA-3 

36 

35, 665 

-4. 974 

1. 0  X  10 

A 

38 

35,700 

-4. 991 

401,  559 

1.0  X  10" 

For  all  practical  purposes,  the  38th  iteration  is  an  optimum  trajectory  having 
maximum  terminal  velocity  for  the  specified  terminal  values  on  altitude  and 
flight-path  angle. 

Figure  10  shows  h,  y ,  V,^,,  and  a  vs  t  for  the  nominal  and  optimum 
trajectories.  The  angle -of- attack  program  for  the  optimum  trajectory  is  a 
modest  program,  starting  from  0  decreasing  slowly  to  -7  degrees  tod 
increasing  to  2  degrees  before  falling  off  slowly  to  -17.  5  degrees  at  the  end  of 
burnput  of  the  three-stage.  A  maximum  altitude  of  630, 000  ft  occurred  370  sec¬ 
onds  after  launch  for  the  optimum  trajectory. 
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Figure  10  -  h,  y ,  Vip  and  or  vs  t  for  the  SORTIE  Configuration 
for  Msucimum  Re-Entry  Velocity  for  Terminals  Conditions  of 
Altitude  of  400,  000  ft  and  Flight- Path  Angle  of  -5* 
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This  example  has  demonstrated  how  efficiently  and  effectively  the  trajec¬ 
tory  optimization  procedure  can  determine  the  optimum  trajectory,  maximizing 
the  terminal  velocity  and  meeting  the  specified  terminid  conditions,  even  though 
the  terminal  constraints  on  the  nominal  trajectory  differed  by  an  order  of  mag¬ 
nitude  from  the  specified  values. 
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8.  SORTIE  GLIDING  RE-ENTRY  WITH  A  STAGNATION-POINT 
HEATING  RATE  IN-FLIGHT  CONSTRAINT 


The  second  part  of  the  demonstration  example  considered  the  problem  of 
determining  the  re-entry  glide  trajectory  with  minimum  stagnation-point  heating 
rate  penalty  function,  P.  for  the  SORTIE  vehicle.  (See  Section  3.  2  for 
details  on  in-flight  constraints.)  This  example  has  demonstrated  the  trajectory 
optimization  computer  program  for  gliding  vehicles  with  an  in-flight  constraint 
for  the  following  initial  conditions: 

34, 000  ft/ sec 
-5* 

400,  000  ft 
70* 

-60* 

70* 

One  of  the  conditions  of  the  re-entry  trajectory  was  that  the  vehicle  reenter 
within  the  Atlantic  Missile  Range;  thus,  trajectories  which  require  more  than 
one  pass  over  the  range  were  not  considered.  The  re-entry  angle  of  -5  degrees 
was  determined  from  a  series  of  trajectories  in  which  the  re-entry  angle  was 
varied  for  a  fixed  re-entry  velocity  of  34,  000  ft/sec.  The  re-entry  velocity  of 
34,000  ft/sec  has  been  specified  by  ASD  as  a  possible  lower  limit  for  the  SORTIE 
test  vehicle. 

The  re-entry  angle  (for  fixed  re-entry  velocity  and  altitude)  determines  a 
peak  value  for  the  stagnation- point  heating  rate.  The  flight-path  angle,  y,  is 
principally  dependent  upon  the  centrifugal  and  gravity  forces  acting  upon  the 
vehicle  during  the  initial  phase  of  re-entry.  Aerodynamic  forces  of  drag  and 
lift  do  not  affect  the  vehicle  motion  until  the  vehicle  enters  appreciably  into  the 
atmosphere.  Thus,  during  the  initial  phase  of  re-entry  (approxinmately  the  first 
90  seconds  iii  this  problem),  the  flight- path- angle  program  is  solely  dependent 
upon  the  initial  velocity,  flight-path  angle,  and  altitude.  Therefore,  there  is  an 
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initial  flight  angle  (for  every  initial  velocity  and  and  altitude)  for  which  the  vehicle 
can  just  stay  in  with  the  application  of  maximum  negative  (negative  is  defined  in 
the  -r  direction)  lift.  For  larger  (less  negative)  initial  flight-path  angles  the 
vehicle  will  skip  out  and  a  single -pass  re-entry  within  the  Atlantic  Missile  Range 
is  not  possible.  For  these  initial  re-entry  conditions,  the  peak  value  of  the 
stagnation- point  heating  rate,  q,  will  be  determined  and  will  occur  at  the  point 
where  the  effect  of\lift  enters.  Thus,  associated  with  every  initial  velocity 
and  re-entry  angle,  there  will  be  a  fixed  value  for  q. 

Once  lift  can  be  used  to  alter  the  flight-path  angle,  a  modulated  lift  reentry 
is  possible.  The  minimum  flight-path  angle  for  reentry  in  a  single-pass  tra¬ 
jectory  vas  found  to  be  in  the  neighborhood  of  -5*  for  the  specified  initial  condi¬ 
tions.  This  value  was  selected  as  being  well  within  the  re-entry  capability  of 
the  vehicle,  but  not  so  sensitive  as  to  allow  the  vehicle  to  skip  out  with  small 
changes  in  the  angle -of-attack  program.  The  angle -of-attack  program  for  the 
nominal  trajectory  was  a  positive  constant  angle-of-attack  (maximum  lift)  until 
the  flight-path  angle  reached  a  value  close  to  zero,  at  which  point  the  vehicle’s 
maximum  negative  lift  capability  is  utilized  to  keep  it  from  skipping  out.  This 
negative  lift  is  used  for  the  remainder  of  the  flight,  and  the  bank  angle  is  used  to 
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Figure  11  -  h  V—,  y  and  4  vs  t  for  SORTIE  Re-Entry  which  Miiiimizes 
Stagnation-Point  Heating  Rate  Penalty  Function 
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modulate  the  negative  lift  component.  A  bank-angle  program  to  keep  the  vehicle 
as  near  as  possible  on  the  original  heading  angle  was  used;  howevert  the  vehicle 
did  wander  off  course. 

'  Figure  11  shows  the  velocity,  altitude,  flight-path  angle  and  the  stagnation- 

point  heating  rate  as  a  function  of  time.  The  stagnation  heating  rate  for  the 

2 

nominal  case  peaked  at  371  BTU/ft  sec  205  seconds  after  reentry.  This  shows 
that  the  lift  program  could  be  improved  upon  as  well  as  reduce  the  value  of  the 
penalty  function  for  stagnation-point  heating  rate  from  a  value  of  29.  2.  The  pen- 
alty  function  level,  was  set  at  186  BTU/ft  sec,  a  value  considerably  below 
the  220  BTU/ft^  sec  peak  reached  at  the  bottom  of  the  first  skip-out  point. 

After  21  iterations,  the  penalty  function  P  was  reduced  to  18.  3  and  the  maximum 
heating  rate  has  dropped  to  302  BTU/ft^  sec.  In  7  additional  iterations, 

was  reduced  to  218  BTU/ft^  sec  and  P  to  a  value  of  1.  q  drops  from  the  peak 
value  of  218  to  approximately  175  and  remains  close  to  this  value  until  the 
velocity  drops  to  about  18,000  ft/sec  and  the  altitude  at  this  point  is  approxi¬ 
mately  180,  000  ft.  From  this  point  on,  the  stagnation- point  heating  rate  drops 
rapidly  as  the  velocity  drops.  A  summary  of  these  computations  are  given  in  Table  2. 

TABLE  2 


SUMMARY  OF  SUCCESSIVE  IMPROVEMENTS  FOR  SORTIE  RE-ENTRY  GLIDE 


Case 

q  -  BTU/ft^‘sec 

^max 

P(q^  =  186  BTU/ft^sec) 

(dP)^/T  -  deg^ 

nominal 

371 

29.  2 

1  X  10"^ 

21 

302 

18.  3 

5  X  10'^ 

28 

218 

•  99 

5  X  10'^ 

Figure  12  shows  the  angle -of -attack  programs  for  these  re-entry  trajec¬ 
tories.  Note  that  the  angle -of- attack  program  has  had  only  minor  changes  for 
the  first  90  seconds  of  flight;  however,  these  small  changes  are  important  for 
they  determine  the  peak  value  for  the  stagnation-point  heating  rate.  The  "spike" 
appearing  in  the  angle -of- attack  program  at  2G0  seconds  appears  to  be  due  to  an 
error  in  input  for  the  angle  of  attack  in  the  nominal  program.  The  value  used 
in  the  nominal  program  was  +50  degrees  and  the  final  trajectory  has  smoothed 
out  this  value  to  about  -8  degrees.  The  angle-of-attack  changes  are  again 
modest,  though  important.  The  bank-angle  program  is  presented  in  Figure  13. 
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Figure  12  -  a  vs  t  for  SORTIE  Re-Entry  which  Minimizes  Stagnation -Point 

Heating  Rate  Penalty  Function 


Figure  13  -  a  vs  t  for  SORTIE  Re-Entry  which  Minimizes  Stagnation-Point 

Heating  Rate  Penalty  Function 
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The  nominal  a  program  was  modulated  to  minimize  vehicle  wander  from  the 
original  heading.  The  optimum  bank- angle  program  has  had  a  significant 
change  over  the  nominal  one,  especially  during  the  tirne  period  of  180  to  230 
seconds.  This  time  period  corresponds  to  peak  stagnation- point  heating  rate 
for  the  nominal  trajectory.  Note  that  the  bank  angle  can  be  used  to  modulate 
the  component  of  lift  in  the  plane  of  the  velocity  and  radius  vectors  to  maintain 
the  desired  relation  of  V,p  and  y  to  minimize  the  stagnation-point  heating 
rate.  Since  q  peaked  during  this  period  on  the  nominal  trajectory,  the 
assumed  bank- angle  program  was  not  a  good  one  and  large  changes  in  this 
program  could  have  been  expected  to  reduce  P  to  a  minimum  value.  To 
reduce  the  peak  q  ,  the  bank  angle  w'ould  have  to  be  changed  such  that  the 
flight-path  angle  decreased  as  the  velocity  dropped.  The  velocity  drop 
depends  upon  the  drag  of  the  vehicle,  and  thus,  high  values  of  angle  of  attack 
are  needed  to  achieve  maximum  velocity  changes.  The  increment  of  lift 
needed  to  change  the  flight-path  angle  a  desired  amount  is  achieved  by  the 
proper  bank  angle  and  bank-angle  modulation  determines  the  minimum  value 
for  the  heating- rate  penalty  function. 


9.  RECOVERY  AREA  FLIGHT  BOUNDARY  FOR  SORTIE 


The  recovery  area  for  the  SORTIE  vehicle  has  been  determined  for  the 
initial  condition  given  by 


II 

0 
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15,  282  ft/ see 

h 

151,617  ft 

o  0 

II 

-7.29* 

= 

83. 96* 

II 

0 

81.  31'  (latitude  =  8,  69 “) 

-32. 34' 

The  initial  conditions  were  arbitrarily  selected  from  the  re-entry  trajectory 
given  in  Section  8. 

Three  optimum  trajectories  were  calculated  for  these  initial  conditions: 
maximum  downrange,  9  ,  maximum,  and  minimum  lateral  latitude,  v.  The 
stopping  condition  was  altitude,  h^  =  1 ,  000  ft.  Figure  14  shows  these  trajec¬ 
tories  on  a  geographic  Mercator  projection  and  a  ’’footprint”  is  sketched  through 
the  three  points.  Figure  15  shows  the  same  trajectories  on  a  Mercator  pro¬ 
jection  of  the  auxiliary  coordinate  system  /li  vs  p,  and  Figures  16  and  17  show 
the  control  function  histories  a  and  a  vs  t  for  the  three  extremal  trajectories. 

The  angle  of  attack  for  the  maximum  downrange  trajectory  is  close  to  the  value 
for  maximum  lift-to-drag  ratio,  namely  a  =  17.  5®  . 

The  maximum  and  minimum  lateral  latitude  trajectories  use  essentially  the 
angle  of  attack  for  maximum  lift-to-drag  ratio.  The  bank-angle  program  reaches 
values  slightly  less  than  50  degrees  and  then  gradually  tapers  off  to  zero  when  the 
turn  angle  approaches  90®.  For  comparison  purposes,  trajectories  with  constant 
bank  angles  were  run  and  the  dashed  footprint  in  Figures  14  and  15  show  the 
results  of  these  runs.  All  of  the  constant-bank-angle  trajectories  show  a  spiral¬ 
ling  tendency  near  the  terminal  portion  of  the  flight.  Two  other  trajectories  are 
shown  for  the  case  of  a  =  ±45®,  in  which  cases  the  bank  angle  was  set  equal  to 
zero  when  the  turn  angle  reached  90®.  These  two  trajectories  were  used  as  the 
nominal s  to  determine  the  maximum  and  minimum  lateral  latitudes.  Figures 
18  through  20  show  the  altitude,  velocity  and  flight -path  angle  as  a  function  of 
time  for  the  three  sets  of  trajectories. 
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Figure  14  -  Geographic  Mercator  Projection  of  Atlantic  Missile  Range  Flight 
Boundary  for  the  SORTIE  Reentry  and  Recovery 
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Figure  15  -  Auxiliary  Mercator  Projection  of  Atlantic  Missile  Range 

Flight  Boundary 
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Figure  16  -  a  vs  t  for  Maximum  Downrange  in  Recovery  Area 
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Figfuire  17  -  a  and  a  vs  t  for  Maximum  and  Minimum  Latitude  in  Recovery  Area 
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Figure  19  •  h,  and  Y  vs  t  lor  Maximum  Lateral  Latitude  in  Recovery  Area 
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Figure  20  -  h.  and  y  vs  t  for  Minimum  Lateral  Latitude  in  Recovery  Area 
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The  flight  boundary  near  the  initial  point  was  not  investigated  since  structural 

and  other  vehicle  limitation  data»  which  would  determine  the  trajectory,  were  not 

available  at  the  time  of  this  investigation.  The  maximum  downrange,  9  ,  was 

determined  from  the  nominal  trajectory  for  the  angle  of  attack  for  maximum  lift- 

to-drag  ratio  of  17.  5  degrees  and  a  bank  angle  equal  to  zero.  The  nominal  tra- 

jectory  had  a  range  of  1008  nmi  (0  =  16.  8  degrees)  and  the  maximum  downrange 

>;<  - 

was  1026  nmi  (0  =  17.  2  degrees),  an  increase  in  range  of  about  18  nmi  or  about 

a  2%  increase.  Classical  use  of  (L/D)^^^  angle  of  attack  gives  a  good  approxi¬ 
mation  for  the  nominal  trajectory  to  be  used  to  determine  maximum  range  capa¬ 
bilities,  Maximum  and  minimum  lateral  latitude,  v  and  -v  respectively,  were 
equal  to 


V 

max 


V 

min 


6.  59® 


-6.60® 


which  were  approximately  8.  2%  increase  over  the  nominal  values.  Table  3 
shows  the  summarized  results. 


TABLE  3 

MAXIMUM  AND  MINIMUM  LATERAL  LATITUDES  IN  RECOVERY  AREA 


V  .  _  -  deg 

nominal  ® 

V  -  dee 

max  ® 

Av  -  deg 

Av 

^nominal 

6.  10 

6.  59 

.49 

,0816 

-6.  10 

-6.  60 

50 

.0820 
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10.  RESULTS 


A  demonstration  problem  has  determined  certain  of  the  performance 
boundaries  for  a  typical  three -stage  booster  system  and  the  SORTIE  vehicle 
during  reentry  and  recoveryi  Three  phases  of  the  computer  program  developed 
to  solve  three-dimensional  trajectory  optimization  problems  were  demonstrated. 
These  phases  included  the  determination  of  trajectories  for  1)  vehicles  with 
thrust,  2)  re-entry  glide  vehicles  with  an  in-flight  constraint  and  3)  lateral 
maneuvers  for  a  gliding  vehicle.  Starting  from  a  nominal  trajectory,  each 
series  of  trajectories  was  computed  using  the  steepest-ascent  computational 
procedure.  This  computational  procedure  has  shown  that  the  method  can 
determine  in  an  efficient  and  in  an  effective  manner  the  optimum  trajectory 
satisfying  not  only  the  specified  terminal  conditions,  but  in-flight  constraints 
as  well.  The  efficiency  of  the  computer  program  depends  upon  the  ingenuity  of 
the  engineer  and  the  nominal  trajectory  selected.  However,  even  with  the 
selection  of  a  nominal  which  does  not  satisfy  too  closely  the  desired  terminal 
conditions,  the  computer  program  is  able  to  determine  in  a  very  short  time 
the  optimum  trajectory. 

The  computer  programs  can  greatly  aid  the  engineer  in  solving  difficult 
trajectory  optimization  problems.  Careful  analysis  of  each  set  of  runs  by  the 
engineer  can  also  point  the  way  for  a  better  nominal  to  be  used  in  obtaining  the 
optimum  solution  in  fewer  iterations. 

These  computer  programs  are  especially  valuable  for  the  determination  of 
solutions  to  problems  with  in-flight  and  terminal  constraints.  This  study 
indicates  that  areas  where  significant  gains  can  be  achieved  occur  during  the 
flight  when  the  constraints  are  the  overriding  considerations  of  the  problem. 
Thus,  for  vehicles  reentering  the  earth's  atmosphere  at  near  orbital  speeds 
and  greater  and  during  high-speed  maneuvers  when  the  angles  of  attack  and 
bank  angles  may  become  quite  large,  the  in-flight  constraints  play  a  very 
important  role  in  determining  the  control- variable  programs.  Optimum  paths 
in  this  region  of  flight  lie  in  a  very  narrow  band,  and  the  control  progranris 
must  be  exact  if  the  specified  conditions  of  the  problem  are  to  be  met.  This 
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has  been  shown  by  both  example  s  of  the  thrust -direction  program  for  a  typical 
three -stage  booster  system  and  the  SORTIE  re-entry  glide  with  stagnation - 
point  heating  rate  in-flight  constraint. 

Control  programs  determined  by  the  cut-and-try  approach  do  not,  in 
general,  satisfy  the  constraints  imposed  upon  the  problem  and  to  satisfy  all 
the  constraints  simultaneously  would  be  impractical  except  for  the  fortuitous 
exceptional  problems.  The  steepest-ascent  technique  can  determine  these 
control  variable  programs  in  an  efficient  and  effective  manner;  starting  with 
a  reasonable  nominal  contral  program  each  iteration  approaches  closer  to  the 
optimum,  eventually  satisfying  simultaneously  the  imposed  constraints  and 
extremilizing  the  payoff  quantity.  This  technique  has  been  shown  to  be  espe¬ 
cially  valuable  in  solving  complex  optimum  programming  problems. 
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APPENDIX 


THRUST  AND  AERODYNAMIC 
COMPONENTS  TRANSFORMATIONS 

Let  the  x,  y,  z  axis  eystem  be  aligned  in  the  vehicle  as  shown  in 
Figure  21,  with  the  two  thrust  vector  offset  angles  of  6,p  and  i^.  The 
components  of  thrust  along  the  three  axes  are  given  as  follows: 

Tjj  ■  T  cos  i^  cos  6^ 

Ty  *  T  cos  i^  sin  6^ 

Tj,  »  T  sin  i^ 

where  the  magnitude  of  the  thrust  is  given  by  T, 

T  =  F(t)  +  (p  -  p  )  A 
c  oo  c 


(1) 


(2) 


Figure  21  -  Thrust  Vector  Orientatibh 
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F(t)  is  a  given  function  of  time  and/or  stages  and  is  the  basic  thrust  usually 
given  for  sea  level  pressure  conditions.  The  second  term  of  the  equation 
represents  the  effect  of  atltitude  in  the  basic  thrust.  Where  P  is  ambient 
pressure  at  sea  level  or  reference  altitude,  P  is  the  ambient  pressure  at 

iO 

altitude  and  A  is  the  reference  exit  area  associated  with  the  rocket  engine 
e 

for  each  stage. 

The  components  of  the  thrust  vector  given  in  body  axis  components  must 
be  transformed  into  components  in  the  geographic  coordinate  system  to  be  usee 
in  the  optimization  program.  The  overall  transformation  can  be  obtained  as 
a  product  of  four  transformations,  each  one  representing  a  rotation  about  a 
single  axis. 

The  first  rotation  is  about  the  body  y  axis  through  the  angle -of- attack  or: 


*  A  * 
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0 

0 

1 


cos  a 
-sin  a 
0 


A 

X 

A 

y 

A 

Z 


(3) 


where  the  hatted  symbols  represent  unit  vectors.  This  transformation 
rotates  the  body  z  and  x  axes  into  the  lift  direction,  L,  and  the  velocity 

A 

direction, 

The  second  rotation  is  about  the  velocity  vector  through  the  bank- angle 
<7 : 
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This  transformation  rotates  the  lift  axis  and  the  body  y  axis  into  the  y  and 
^  directions,  where  Y  lies  in  a  vertical*  plane  defined  by  the  radial  direction 
r,  and  the  velocity  direction  '0—.  The  $  vector  thus  lies  in  a  horisontal  plane. 

The  third  rotation  is  about  the  p  vector  through  the  flight  path  elevation 
angle*  y: 
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A  ^  A  ^ 

This  transformation  rotates  the  y  and  vectors  into  the  r  and  vectors 

A  ^  “ 

where  r  is  the  direction  of  the  vertical  and  is  the  direction  of  the 
horizontal  component  of  the  velocity. 

A 

The  fourth  and  last  rotation  is  about  the  radial  vector  r  through  the  flight 
path  heading  angle,  p: 
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This  transformation  rotates  the  and  p  vectors  into  the  B  and  ^  vectors  of 
the  geographic  coordinate  system.  - 

The  overall  transformation  matrix  is  therefore  given  by  the  product  of 
the  four  transformation  matrices  of  equations  III-6,  5,  4,  and  3: 
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Due  to  the  earth’s  oblateness  f  is  not  quite  the  vertical  direction  (See,  for 
example,  Reference  1). 
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where  the 


where 


a.  .'e  are  given  below.  The  <^re  as  follows : 
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The  aerodynamic  force  components  may  be  written  as 


A 

A  = 


A  A 

LL  -  DV^. 


Hence,  the  components  in  the  geographic  coordinate  system  are  given  by 
the  product  of  the  three  rotations  of  equations  III-6,  5,  and  4: 


• 

■ 

A 

r 

hi 

hz 

L 

hi 

^22 

-D 

1 - 

cr 

hzj 

where 


hi 

s 

cos  <r  cos  y 

hz 

= 

sin  y 

hi 

=' 

-  sin  y  cos  0  cos  S'  -  sin  ^  sin  a 

hz 

cos  y  cos  0 

^31 

- 

cos  /3  sin  O'  -  sin  y  sin  0  cos  a 

hz 

= 

cos  y  sin 

(10) 


238 


REFERENCES 


(1)  Brown,  R.  C.  ,  Brulle,  R.  V.  ,  and  Giffin,  G,  D,  ,  Six -Degree -of- Freedom 
Flight"Path  Study  Generalized  Computer  Program,  W ADD  Tecfehic^l  Re¬ 
port  60-781,  Part  I,  May,  19^1. 

(2)  Kelley,  Henry  J.  ,  Kopp,  Richard  E.  ,  and  Mayer,  H,  Gardner,  Succes- 
sive  Approximation  Technique  for  Trajectory  Optimization,  Proceedings 
of  the  IaS  Symposium  on  Vi^icle  System  Optimization,  Garden  City,  New 
York,  November  28-29,  1961,  pp.  lG-25. 

(3)  Konecci,  E.  B.  ,  Manned  Space  Cabin  Systems.  Chapter  in  ‘'Advances 

in  Space  Science,  "  Vol.  1  (edited  by  Or  away).  Academic  Press,  1959. 

(4)  Hankey,  Wilbur  L.  ,  Jr,  ,  Neumann,  Richard  D.  ,  Flynn,  Evard  H.  , 

Design  Procedures  for  Computing  Aerodynamic  Heating  at  Hypersonic 
Speeds,  WA&C  Technical  Report  59-610,  June  196(). 

(5)  Kelley,  H.  J.  ,  Gradient  Optimal  Flight  Paths,  Jour,  Amer. 

Rocket  Soc,  ,  Vol.  30,  pp.  9?V-^547  OctoSer  196Cr 

(6)  Bryson,  A.  E.,  Denham,  W.  F,  ,  Carroll,  F.  J.  ,  and  Mikami,  K,  ,  Lift 
or  Drag  Programs  that  Minimize  Re-entry  Heating',  Jour.  Aerospace 
Sciences,  Vm,  29,  No.  4,  April  2. 

(7)  Bryson,  A,  E.  ,  and  Denham,  W.  F.  ,  A  Steepest -Ascent  Method  for 
Solving  Optimum  Programming  Problems,  Raytheon  Report  BR-1303, 

1961 ,  and^our.  AppL  Mech.  ,  Vol.  2^,  No.  I,  March  1962. 

(8)  Bryson,  A.  E.  ,  Mikami,  K.  ,  and  Battle,  C.  T.  ,  Optimum  Lateral  Turns 
for  a  Re-entry  Glider,  Aerospace  Engineering,  Vol.^lT,  l^o,  3,  March 

1962.  Also  Raytlieon  Report  BX- 110. 

(9)  Koelie,  H.  H.  ,  Editor,  Handbook  of  Astronautical  Engineering,  McGraw- 
Hill  Book  Company,  IncTi  1961^ 

(10)  Mikami,  K,  ,  Battle,  C.  T.  ,  Goodell,  R.  S.  ,  and  Bryson,  A.  E.  ,  Three- 

PiTnensional  Trajectory  S^dy,  Part  I  -  Optimum  Program- 

ming  Formulation,  A5D-^t)Il^6^-295,  t^art  I,  May,  1962. 

(11)  Mikami,  K.  ,  Battle,  C.  T.  ,  Goodell,  R,  S.  ,  and  Bryson,  A.  E.,  Consultant, 
Three-Dimensional  Trajectory  Optirnization  Study,  Part  II,  Computer 
Program  and  User's  Manual,  ASD-TDR -62 -295,  Part  II,  May,  1962. 

(12)  Mikami,  K.  ,  Battle,  C.  T.  ,  Goodell,  R.  S.  ,  and  Bryson,  A.  E.,  Consultant, 
(U)Three -Dimensional  Trajectory  OpHmization  Study,  Part  III,  Demonstration 

Problem  Report^  ASD-TDR-6i-2v5,  Part  III,  May,  1962. 

CON  FIDE  N  T  lA  L  REPORT 


ASD-TDR-63-119 


The  Doliac  Macro-Micro  Control  Logic, 
Its  Synthesis,  Evaluation,  Potential  and  Problems 

By 

Daniel  O,  Domrnasch 


DODCO.  Inc. 

Blawenburg,  New  Jersey 


241 


THE  DOLIAC  MACRO-MICRO  CONTROL  LOGIC,  ITS  SYNTHESIS, 
EVALUATION,  POTENTIAL  AND  PROBLEMS 

Daniel  O,  Dommasch,  DODGO,  INC, 

Abstract 

The  acronym,  DOLIAC,  is  formed  from  the  first  letters  of  the  title, DODCO 
OPTIMUM  LIMITED  INFORMATION  ADAPTIVE  CONTROLLER.  The  term  macro- 
micro  signifies  that  both  the  over-all  guidance  function  (macro  control)  and  the 
autopilot  function  (micro  control)  are  accomplished  within  the  same  logical 
optimized  system. 

The  DOLIAC  logic  concept  of  autopilot  system  optimization  was  developed  by 
DODCO,  INC. ,  in  1958,  and  subsequently  has  been  examined  in  detail  by  the 
Cornell  Aero  Lab,  and  other  investigators,  References  pertaining  to  the  logic 
are  provided  in  the  paper.  The  macro-micro  additions  to  the  logic  were 
accomplished  early  in  i960  and  have  since  been  subject  to  continued  research 
and  development  effort.  The  basic  DOLIAC  logic  is  obtained  using  either 
standard  differential  or  variational  calculus  procedures  and  both  the  necessary 
and  sufficient  conditions  for  solution  existence  are  satisfied  by  the  analysis 
(which  is  based  on  the  use  of  an  inequality  boundary  constraint  on  system 
action) ,  The  research  leading  to  the  development  of  the  DOLIAC  logic  has 
been  conducted  under  Air  Force  contracts  directed  by  Mr.  P,  C.  Gregory  of 
the  Flight  Controls  Laboratories,  ASD. 

It  is  demonstrated  that  the  Lagrange  mult4)lier  acting  on  an  inequality  restraint 
performs  the  same  fimction  as  the  slack  variable  in  linear  programming  analyses 
and  that  the  basic  DOLIAC  logic  defines  a  simple  non-linear  gain  switching  system 
requiring  error  and  error  rate  data  for  its  functioning.  Since  system  perforni- 
ance  depends  on  the  direct  sensing  of  error  rates,  adequate  sensing  of  these 
rates  must  be  possible  or  system  performance  is  sharply  degraded.  Thus,  there 
is  no  point  in  employing  the  DOLIAC  logic  to  try  to  provide  a  control  precision 
which  is  beyond  the  precision  of  possible  error  measurements. 

Although  simple  heiuristically  based  system  stability  criteria  for  the  DOLIAC 
logic  are  presented  in  the  paper,  these  criteria  are  known  to  be  too  restrictive, 
and  much  work  remains  to  be  done  in  the  area  of  analytic  stability  analysis. 
Although  the  non-linearities  of  the  DOLIAC  logic  are  of  an  apparently  simple 
sort,  their  nature  precludes  the  definition  of  an  equivalent  linear  system  and, 
therefore,  precludes  the  direct  use  of  existing  linear  stability  criteria  for 
analysis.  Whereas  the  anal)d;ic  treatment  of  DOLIAC  stability  is  incomplete, 
extensive  numerical  studies  on  both  digital  and  analog  computing  equ4>ment 
reveal,  without  question,  the  superiority  of  the  non-lihear  cptimized  logic 
over  conventional  linear  system  performance  in  niany  cases.  In  other  cases. 
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the  DOUAC  theory  may  be  used  to  demonstrate  that  linear  error  and  error 
rate  systems  themselves  form  a  degenerate  class  of  optimum  systems  which, 
in  these  cases,  may  represent  the  only  practical  optimtuns  obtainable. 

It  seems  apparent  that  tiie  mere  existence  of  the  DOUAC  logic  provides  an 
affirmative  answer  to  the  question  "Is  optimum  syi>thesis  practical"';  how¬ 
ever,  it  is  shown  that  the  DOUAC  theory  is,  in  and  of  itself,  still  rather 
crude,  representing  only  a  single  step  forward  from  conventional  linear 
theory  and  there  exists  the  potential  for  improvements  in  the  future. 
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UST  OF  SYMBOI5 
Restraint  limit 

error  quantity  (desired  -  actual  value) 

error  at  point  n,  time  t 

error  at  point  n  +  1,  time  t  +  At 

error  at  point  n  +  2,  time  t  +  2  A  t 

controlled  error  at  point  n  +  1 

uncontrolled  error  at  point  n  -t-  l 

error  rate 

error  acceleration 

commanded  error  acceleration 

function  to  be  made  stationary 

stationary  value  of  f  along  root  locus  curves  in  plane  of 
f  and  commanded  error  acceleration 

Level  line  value  of  f 

Equation  of  ray  in  the  /  ,  A  e  plane  joining  the  maximum 

c 

and  miniinum  solutions  of 
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LIST  OF  SYMBOLS  (C6ilt’ d) 

Inverse  gain  factor 

Forward  loop  gain  factor 

derivative  level  of  Taylor  expansion,  Lagrange  multiplier  ratio 
as  a  subscript,  point  in  time 
response  factor 
time 

prediction  or  lead  time 

sample  time  of  sample  data  system 
pitch  attitude 
Lagrange  multiplier 


RestraiiA  function 


THE  DOLIAC  MACRO-MICRO  CONTROL  LOGIC,  ITS  SYNTHESIS,  EVALUATION, 
POTENTIAL  AND  PROBLEMS 

1.)  INTRODUCTION 

The  word  "optimum,"  in  common  with  such  words  as  "logical"  and  "objective," 
is  normally  subjective  in  its  connotations  to  any  given  individual.  Thus,  most 
people  consider  themselves  as  logical  and  objective  wdiich  is,  of  course, 
paradoxical,  since  any  consideration  of  oneself  is,  by  definition,  subjective. 

The  fact  that  a  similar  paradoxical  situation  exists  concerning  the  word 
"optimum"  requires  us  at  the  outset,  if  we  want  to  deal  with  specifics  rather 
than  generalities,  to  limit  our  concept  of  (^timum  in  such  a  way  that  we  can 
obtain  a  mathematical  statement  capable  of  defining  what  we  mean  by  "optimum." 
We  must,  furthermore,  limit  ourselves  to  consideration  of  practical  (i.e. , 
physicaUy  obtainable)  optima  which  are  respectful  of  the  gotreming  laws  of 
physics,  otherwise  we  are  likely  to  obtain  solutions  to  problems  which  exist 
abstractly  but  vdiich  have  no  real  physical  counterparts.  These  restrictions 
on  acceptable  methods  of  synthesis  are  not  peculiar  to  any  field  of  analysis, 
nor  are  they  in  any  sense  new,  novel  or  original.  Indeed,  the  need  for 
restricting  optimum  solutions  to  obey  the  laws  of  nature  was  recognized  long 
ago  by  both  Euler  and  Lagrange  (the  historical  fathers  of  mathematical 
optimizing  procedures)  and,  as  a  matter  of  fact,  the  most  widely  used  method 
of  introducing  restraints  of  a  physical  nature  into  maximum-minimum  problems 
carries  the  name  of  Lagrange. 

In  the  event  that  we  are  able  to  completely  define  the  nature  of  a  restraint 
either  in  the  form  of  a  differential  equation  (e.  g. ,  when  applying  the  laws 
of  motion)  or  in  some  other  specific  way  (e.g. ,  isoperimetric  restraints), 
the  ^proach  to  problem  solution  using  the  method  of  undetermined  Lagrange 
multipliers  is  rather  formalized  although  it  is  necessary  to  properly  inter¬ 
pret  the  nature  of  the  restraints.  On  the  other  hand,  if  the  restraints  are 
not  so  well  defined,  as  in  the  case  of  inequality  conditions,  then  there  may 
be  room  for  argument  as  to  the  applicability  of  the  particular  method  of 
employing  a  given  restraint  and  as  to  the  meaning  of  the  results  obtained 
from  a  given  application  procedure.  For  the  class  of 

to  discuss,  the  restraints  are  of  the  inequality  type,  and  the  lA;gi'ange  multi¬ 
plier  takes  on  some  of  the  characteristics  of  the  "elack"  variable  encountered 
in  the  theory  of  linear  programming.  Thu.'',  the  physical  s^ificanoo  of  the 
multiplier  is  somewhat  obscure,  and  this  is  one  of  the  reasons  why  ihe  new 
logic  represents  only  a  basic  first-s'iep  in  improvement  over  simple  linear 
error  and  error  rate  feedback  control  theory.  R  does,  however,  represtsPt 
an  improvement  and  it  does  work.  Although  the  basic  DOLIAC  logic  is  non¬ 
linear  in  nature,  in  a  degenerate  form  it  includes  the  entire  set  of  linear 
error  and  error  rate  feedback  systems  as  a  subset  which  is  extremely  interest¬ 
ing  since  this  offers  some  proof  of  the  thought  that  linear  feedback  may  con¬ 
stitute  an  optimum  solution  to  specific  (if  restricted)  classes  of  problems. 
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With  these  introductory  thou^s  in  mind»  we  now  seek  to  define  a  practical 
optimum  criterion,  and  to  do  this  we  need  to  ask  just  what  actually  can  be 
accomplished  by  a  real  control  system. 


In  general  sense,  any  control  logic  system  seeks  to  control  some  variable 
so  that  the  error,  defined  as  the  difference  between  a  desired  value  of  the 
variable  and  the  existing  value,  vanishes  at  all  points  and  for  all  time.  In 
a  (^amic  problem,  this  is  an  impossible  assignment— unless  we  are  willing 
to  hedge  a  bit  on  the  meaning  of  the  term  ’’vanish,  ”  so  that  practically  speak¬ 
ing  we  can  only  require  that  the  error  be  minimized  on  an  absolute  or  magnitude 
basis.  We,  therefore,  have  the  practical  requirement  that  we  should  seek  the 
minimum  possible  numerical  value  of  error  regardless  of  sign,  and  this  is  a 
requirement  which  is  meaningful  mathematically.  We  need  to  acknowledge  the 
fact  that  all  systems  have  inertial  of  one  form  or  another  so  that,  in  the  absence 
of  infinite  command  authority,  no  instantaneous  changes  in  system  displacement 
state  are  possible.  This  means  that  there  is  nothing  we  can  do  to  instantaneously 
change  the  displacement  error  pattern,  but  that  we  must  wait  a  finite  time  for 
control  action  to  have  its  effect  on  an  existing  error  state.  This  is  a  physical 
limitation  which  cannot  be  circumvented,  but  rather  must  be  accounted  for, 
by  modifying  our  criterion  to  state  that  alt  we  can  hope  to  accomplish  is  a 
minimization  of  absolute  error  sometime  in  the  fxiture,  with  this  future  time 
depending  on  the  command  authority  at  our  disposal.  In  the  absence  of  a  change 
in  command,  and  considering  only  a  small  time  increment,  At^,  we  may  relate 

the  state  of  error  in  the  future  to  the  present  error  state  via  a  Taylor  e^ansion 
about  a  present  time  point  ”n,”  i,e, ,  we  may  write  that 


nc  n  p  n  2  n 


ml 


(  I  :  1 ) 


In  the  presence  of  external  disturbances  which  are  random  in  nature,  the 
higher  derivatives  are  imcertain  and,  moreover,  although  we  cannot  instan¬ 
taneously  alter  e  ,  we  can  produce  an  almost  instantaneous  change  in  the 

n  ■  ' 

error  acceleration  e  .  These  factors  together  prevent  the  use  of  the  full 
n 

Taylor  expansion  as  an  accurate  representation  of  the  uncontrolled  error 
history  and,  therefore,  the  series  must  be  truncated  at  the  e^  level  (as  the 

hi(^est  order  term)  and  At^  restricted  in  magnitude  accordingly. 

If  we  now  postulate  that  a  conimand  is  generated  on  the  acceleration  level, 
then,  if  the  command  is  kAe  where  k  is  an  inverse  gain  factor,  we  have,  as 
a  matter  of  definition^  ° 
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( 1 :  2  ) 


(en+i 


^nc 


If  there  were  no  restraints  imposed  on  the  operation  of  the  system,  we  could 
require  that  (e  )  ,  the  error  at  time  t  +  At  in  the  presence  of  a  command, 

n  T  X  c  p 

be  zero.  In  this  event,  the  error  command  would  be 


{ 1 :  3  ) 


in  which  it  is  apparent  that  Z/kAt^^  is  equivalent  to  the  conventional  forward 
loop  gain  factor  which  we  will  designate  simply  as  K,  Thus, 


K  =  2/kA+ 


( I  :  4  ) 


and 


nc 


(  I  :  5  ) 


If  we  now  limit  ourselves  to  simple  error  and  error  rate  systems,  then, 
from  (1:1) 

^en+l^nc  =  «n  +  ^Vn  f'^S) 

so  that 


( 1 :  7  ) 


This  is  the  conventional  equation  of  a  fixed  gain  error-and  error  rate  feed 
back  controller,  and  it  is  important  to  note  that  the  equation  is  the  result 
of  our  making  the  idealized  assumption  that  control  authority  exists  to 
properly  implement  a  desired  command.  This  assumption  is  seldom  correct 
in  practice,  yet  equation  7,  when  K  is  properly  selected,  frequently  yields 
satisfactory  control.  We  will  return  to  this  point  later  on  in  the  paper. 
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In  conventional  linear  theory,  the  term  At^  is  referred  to  as  a  "damping" 

factor.  Such  usage  follows  by  analogy  mth  the  equation  of  motion  of  a  linear 
system  wherein  the  coefficient  of  the  rate  term  represents  the  systems 
equivalent  fluid  damping  constant.  This  usage,  althou^  conventional,  is 
inexact  and  possibly  misleading  when  we  use  equation  7  to  control  a  system 

having  non-linear  dynamics.  As  a  matter  of  fact.  At  is  properly  referred 

P 

to  as  a  "prediction  time, "  although  the  term  "lead  time"  is  also  applicable. 

Since,  historically,  direct  application  of  equation  7  is  known  to  provide 
adequate  results,  one  must  suppose  that  the  crudeness  of  its  development 
is  justified  by  the  fact  that  it  works,  but  by  the  same  token  it  must  also  be 
supposed  that  improvements  can  be  made  without  too  much  trouble,  and  that 
these  improvements  need  not  be  sophisticated  to  provide  substantial  gains 
in  performance. 

The  first  improvement,  which  is  more  or  less  obvious  ought  to  be  made, 
is  to  provide  information  in  some  form  or  other  that  the  command  authority 
is  limited.  This  statement  is  an  inequality  restraint  on  the  problem,  which 
introduces  a  Lagrange  multiplier,  but  being  that  the  restraint  is  indefinite, 
the  multiplier  cannot  be  defined  directly  by  the  optimum  analysis  for  if  it 
were,  then  the  restraint  would  be  isoperimetric  and  control  authority  would 
always  be  set  at  tlie  limiting  value.  In  this  ewnt,  our  problem  would  be 
determined  beforehand  and  optimiz^idn  would  be  impossible.  This  use  of 
the  Lagrange  muUiplier  as  a  method  of  taking  ig)  the  "slack"  in  the  inequality 
restraint  is  analogous  to  the  introduction  of  a  "slack"  variable  in  linear 
programming  analysis  to  account  for  inequality  type  restrictions  on  the 
objective  fwxction. 

As  we  shall  show,  the  Lagrange  multiplier,  introduced  as  an  indicator  of 
an  inequality  adjoint  condition,  has  an  intimate  relationship  with  the  sys¬ 
tem  gain  and,  therefore,  is  subject  to  all  problems  characteristic  of  the 
gain  selection  process.  There  is,  however,  an  important  difference  which 
should  be  kept  in  mind.  The  introduction  of  a  simple  restraint  renders  the 
logic  nonlinear  and,  therefore,  no  direct  use  can  be  made  of  linear  theory 
for  the  establishment  of  the  proper  value  of  the  multiplier.  Thus,  we 
emphasize  at  this  point  that  despite  the  disarmingly  simple  form  of  the 
equations  describing  the  DOLIAC  logic,  tliat  tiiis  logic  is  intrinsically  non¬ 
linear  and  cannot  be  analyzed  in  general  using  root-locus  procedures,  the 
first  method  of  Liig)punQff,  describing  function  techniques  or  similar  pro¬ 
cedures  u^ich  sometimes  are  useful  in  the  stu<ly  of  "mildly"  ncmlinear 
systems.  On  the  other  hand,  there  are  some  indications  that  conservative 
stability  boundaries  can  rather  readily  be  established  for  the  QOLIAC  type 
logic  on  a  comparative  basis.  We  shail  say  more  about  this  point  later  on 
in  the  paper. 
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To  complete  our  iotroductlon,  we  make  note  of  the  fact  that  since  our 
criterion  for  optimum  performance  Is  to  minimize  numerical  error  at 
some  short  time  in  the  future,  it  is  no  <'.«ossible  to  directly  operate  on 

the  error  and  set  the  derivative  d(e  ,  )  / d Ae  to  0,  since  this  pro- 

n  4  .f  c  c 

cedure  as  a  mathematical  definition  U‘.  cti^pable  of  finding  only  maximum 
(large  plus  values)  or  minimum  (large  tKtgative  values)  of  (e  ^  j)  ^  rather 
than  minimum  numerical  Vitlues. 

This  diEficulty  can  be  avoided  by  seoM;cig  the  stationary  values  of  the 

qiiantity  (e  .  , )  “  {  however,  this  aiodlf sies  the  basic  criterion  in  a 
n+lc 

fashion  which  emphasizes  large  etTOyri  while  de-emphaslzing  small  errors 
and,  accordingly,  althou^  the  error  squared  criterion  produces  satis¬ 
factory  control  in  some  instances,  it  is  inherently  a  poor  regulatory 
criterion.  Evidently,  the  artifice  of  modifying  the  criterion  to  suit  the 
mathematics  is  hardly  desirable.  We  could  directly  seek  the  maximums 
and  minimums  of  absolute  error;  however,  this  involves  some  definition 
difficulties  which  can  be  avoided  by  seeking  the  stationary  values  of  1/  e. 

The  maximums  and  minimums  of  this  inverse  function  represent  the 
closest  approach  of  the  error  to  a  zero  value  regardless  of  sign  of  the 
error  and  permit  the  direct  utilization  of  both  the  maximum  and  minimum 
solutions  which  evidently  must  exist  for  an  arbitrary  function.  It  is  of 
interest  to  note  that  the  use  of  the  error  squared  criterion  effectively 
eliminates  one  class  of  solutions  immediately  and  leads  only  to  the 
conventional  error  and  error  rate  linear  feedback  h3q>othests.  On  the 
other  hand,  the  use  of  an  inverse  error  function  immediately  introduces 
switching  logic  into  the  problem  and,  as  well,  a  requirement  for  the 
selection  of  the  best  of  two  answers  available  in  any  given  case.  We  thus 
obtain,  without  ambiguity,  a  direct  and  proper  solution  to  the  postulated 
optimum  problem. 

In  the  following  section  of  this  paper  we  shall  develop  the  method  of  intro¬ 
ducing  inequality  restraints  into  the  reciprocal  error  stationary  point 
analysis  and  this  will  lead  directly  to  the  basic  DOLIAC  logic  concepts. 

2.  )  SYNTHESIS  AND  ANALYSIS  OF  THE  DOLIAC  LOGIC 

We  have  indicated  that  the  DOLIAC  logic  is  obtained  directly  by  seeking 
the  stationary  points  of  an  inverse  error  function  subjected  to  an  inequality 
restraint  which  limits  the  magnitude  of  possible  control  action.  To  provide 
a  physical  basis  for  the  mathematical  development,  we  need  to  discuss  some 
basic  concepts  sqiplicabte  to  the  problem. 


The  Lagrange  multiplier  technique  of  ordinary  differential  calculus  and  the 
technique  of  linear  programming  (vtdiich  finds  wide  application  in  operations 
research  for  problems  vdiere  linear  restraints  with  ’’corners"  are  involved) 
are  both  based  on  the  same  geometric  considerations.  This  geometry  is  easy 
to  Adsualize  in  a  three-dimensional  problem  space  as  illustrated  by  Figures 
2:1  and  2:2. 


Figure  2:2.  Linear  Programming  Technique 
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Figure  2:1  illustrates  a  standard  Lagrange  mult4>ller  problem  involving 
the  determination  of  the  maximum  and  minimum  values  of  a  fuimtion  subject 
to  the  restraint  that  these  stationary  poiqts  shall  lie  on  a  cylindrical  (not 
necessarily  a  circular  one)  surface  g(x,y)  =  C.  The  geometric  solution  is 
s^parent  from  the  figure.  Figure  2:2  presents  atypical  linear  programming 
problem  showing  a  plane  of  solution  and  a  polygonal  prismatic  restraint  which 
limits  the  problem.  The  maximum  and  minimum  solutions  are  those  corres^- 
ponding  to  the  maximum  and  minimum  values  of  z  on  the  solution  plane  which 
lie  on  the  boundary  of  the  polygonal  cylinder. 

Although  a  three-dimensional  solution  space  has  been  considered  for  illustra¬ 
tion  purposes,  the  same  geometric  concepts  apply  to  an  "n"  dimensional  solu¬ 
tion  space,  even  though  one  cannot  use  a  physical  model  to  illustrate  this 
general  "n"  dimensional  case. 

In  the  theory  of  linear  programming,  iterative  techniques  exist  for  handling 
inequality  types  of  restraint  conditions  in  a  precise  fashion.  The  commonly 
used  procedure  is  to  introduce  so-called  "slack!'  variables  which  transform 
the  inequality  restraints  into  apparent  equalities;  in  other  words,  the  slack 
variables  take  up  the  slack  in  the  inequalities.  The  slack  variables  are 
eliminated  from  the  solution  during  iteration  on  the  problem  matrix. 

No  apparently  comparable  technique  exists  for  handling  inequality  restraints 
for  problems  involving  continuous,  but  nonlinear  functions.  However,  the 
Lagrange  multiplier  itself  may  be  considered  a  slack  variable  which  is 
determined  by  conditions  external  to  the  basic  problem  stipulation. 

The  development  of  the  theory  of  optimum  control  synthesis  methods  involves 
the  use  of  a  simple  inequality  restraint,  vdiich  is  introduced  via  an  indefinite 
Lagrange  multiplier.  The  multiplier  itself  is  not  established  by  the  analysis 
because  the  restraint  is  an  inequality,  and,  indeed,  were  the  restraint  of  any 
other  type  c^imization  would  not  be  possible.  Thus,  the  multiplier  takes  iq> 
the  slack  or  uncertainty  of  the  formulation  and  performs  the  function  of  a 
"trade  off'  parameter  in  the  sense  used  in  game  theory.  AUhou^  the  im¬ 
position  of  the  inequality  restraint  adds  sufficient  information  to  provide 
an  order  of  magnitude  of  improvement  in  control  system  performance, 
further  improvements  should  still  be  obtainable  if  the  physical  action  of  the 
mult4>lier  can  be  established  more  precisely. 

The  particular  problem  we  are  immediately  concerned  with  is  summarized  by 
the  statements: 
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a, 


e  =  error  at  a  point  n 
n 

e  =  error  rate  at  point  n 
n 

e  =  error  acceleration  at  point  n 
n 


then  at  some  point  At^  in  the  future,  (n  +  1) 


^®n+i  ^  =  e  +  e  At  + 

*  nc  n  n  p 

provided  no  command  is  imposed* 

If  a  command  is  generated  at  the  acceleration  level,  then  if  this  command  is 

kAe  where  k  is  an  inverse  gain  factor,  the  error  we  command  at  the  point 
c 

n  +  1  is 

<en+l  =  ^®n+l  *  ''^*c  1"  (  2  •.  2  ) 


we  predict  that  the  error  will  be 


••• 


(2:1) 


We  seek  to  find  that  command  value  which  makes  (e  , )  as  numerically 

(not  algebraically)  small  as  possible  and,  therefore,  seek  the  maxunums  and 

minimums  of  l/(e  ,  ,)  ,  while  recognizing  that  (e  ,)  can  be  zero  in  an 
n  +  rc  n  +  1  c 

unrestrained  case,  and  that  this  solution  will  be  found  unless  the  physical 
restraints  imposed  by  the  systems  (fynamics  are  recognized.  We  give  recogni¬ 
tion  to  these  restraints  by  the  inequality  condition  that  a  command  kAe  must 

c 

be  numerically  less  (regardless  of  sign)  than  some  boundary  value,  C,  i.e. , 

kAe  <  C,  (where  the  sign  of  C  depends  on  whether  Ae  is  plus  or  minus), 
c  c 

In  reciprocal  form,  we  write  that  l/k  Ae  >  C,  where  the  value  of  C  has  been 

c  — 

changed  accordingly*  We  thus  define  a  function  ,  such  that 

«  =  -  c  >  0  (  2 : 3  ) 

kABc 


In  accord  with  the  classical  lagrange  technique,  we  observe  restraM  (2:3) 
by  forming  the  function 
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(214) 


I 


+ 


(en+|) 


+  \(- 


nc 


-  C) 


and  seek  the  stationary  values  of  /,  defined  by  9/  /  9Ae  =  0,  In  this  formu- 

c 

lation  the  constant  AC  vanishes  on  differentiation  so  that  its  sign  is  of  no 
importance;  however,  the  actual  sign  of  C  is  determined  by  the  following  con¬ 
siderations:  If  A  e  is  positive,  inequality  2:3  is  satisfied  as  long  as  A  e  is 
c  c 

sufficiently  small.  However»  ^  A  e  grows>  we  ultimately  find  that  C  dominates, 

c 

and  for  C  being  a  positive  real  number,  we  reach  a  command  limit  on  Ae  .  If 

o 

Ae  is  negative,  C  must  be  negative  and  the  inequality  reversed;  thus,  for  Ae 
c  c 

negative 


0  =  +  c  <  0 

kACc 


12:5) 


and,  in  general 


♦  =  -4:-  ±  C  >  0,  <  0  (2:6) 

kABc 

We  again  recall  that  the  sign  of  C  is  of  no  importance  in  the  classical 
approach,  since  AC  vanishes  on  differentiation. 

In  general,  equation  2;4  may  be  written  as 

/  = - +  XC  ( 2  ;  7  ) 

kAt-  kAo 

t  \  ,  _ P  A**  c 

(en+l 

Accordingly,  /  is  a  function  of  the  single  independent  variable  A  e  ,  and  only 

c 

a  single  minimum  condition  can  be  Imposed  so  that  A.  remains  imdetermined 
by  the  optimizing  process,  i.e. ,  it  depends  on  conditions  external  to  equation 
2:7. 

At  any  given  point  AC  being  a  constant,  serves  merely  to  move  the  entire  function 

curve  upwards  or  downwards  along  the  f  axis  in  a  plane  defined  by  the  orthogonal 

axes  /  and  A  e  ,  and  this  is  the  reason  why  the  term  AC  contributes  nothing  to  a 
c 

maximum-minimum  i?olution. 


253 


If  we  were  to  plot  /  as  a  function  of  (e  ,  , )  ,  we  would  find  that  there  exist 

J  '  n  +  1  no 

two  solutions  for  any  given  combination  of  (e  ,)  ,  X  and  k  with  these 

n  + 1  no 

solutions  being  located  on  either  side  of  a  discontinuity  of  the  function  1/  (e^  ^ 

This  discontinuity  represents  the  unrestrained  soiution  at  vdiich  (e  .)  is  zero. 

n  i  c 

Such  a  solution  is  normally  far  from  desirable  in  a  real  system  >^ich  possesses 
inertia  since  to  obtain  it  for  reason^ly  high  gains,  one  must  force  an  overshoot 
at  a  point  n  +  2,  approximately  as  large  as  the  error  at  the  point  n. 

The  locations  of  the  roots  of  the  optimizing  equation,  by  themselves,  do  not 
fix  the  command,  since  the  command  must  be  selected  from  the  two  available 
possibilities*  The  selection  criterion  is  based  on  the  fact  that  control  is 
applied  steadily  so  that  we  seek  the  minimum  error  at  all  times.  Thus,  if  e^ 

is  the  error  at  n  and  (e^  ^  is  the  predicted  error,  we  select  the  value  of 

A  e  leading  to  the  minimum  numerical  {not  algebraic)  value  of  e  +  (e  . 

The  root  loci  of  the  two  possible  solutions  in  the  /  ,  A  e  space  and  their 

^  c 

movements  relative  to  the  unrestrained  solution  are  of  interest  and  can  be 
defined  by  the  following  analysis* 


Let 


nil  n. 


(2.  8  ) 


and  define  a  ratio  m  by  the  statement 

m  =  X/X  .  (2:9) 

mm, 

so  that  m  varies  directly  with  X.  Also,  let 

K  = 

SO  that  K  varies  inversely  with  k, 
the  optimum  conimand  equation 

tS  *-  _ _ _ 


2/kAt/  (2:10) 

P 

then  the  condition,  8//  dA'e  =  0^  yields 

c 

Xlen+l)-- 

--  _ _ _nc  _  /  ^  •  1 1  % 


which  has  two  roots. 
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For  various  values  of  <e  .  ,)  and  m  and  K  equation  2:11  specifies  the  two 

n  +  1  nc 

root  locations  along  the  A  e  axis.  The  .vertical  dibplacement  of  the  function  / 

c 

at  these  points  is  then  defined  by  the  fact  that. 


/  = 


I _ 

(en+l  )ne 


or  in  terms  of  m  and  k 


/  = 


I 


(en+l )nc 


+  /K 
c 


( 2  :  12) 


and  on  substituting  for  A  e  ,  the  function  along  the  root  locus  curves  f 

C  L. 

is  defined  by 


A 


I 


(en+l ’nc 


\ 

I  , 

+  V'T/nT  -I  ' 


K(en+| ) 

ric 

+  Vl/m'  -I 


(  2 : 13 ) 


which  simplifies  to 


_  I  +  m  +  2-\/tn^ 
^1-=  (en^,)^ 


For  the  cases,  (e  .  ,)  =  ±  1,  K  =  1,  plots  of 

n  +  Inc 


and 


(en+Pc  ^®n+Pnc 


I  fi)  4“ 


(en+j ) 


nc 


(2. 14  ) 


255 


as  functions  of  Ae  ,  appear  as  in  Figure  2:3,  with  equation  2:14  defining 
c 

the  loci  of  the  roots  of  the  control  equations  as  m  varies  from  1  to  “  , 

Note  that  it  can  be  shown  that  the  values  of  m  are  limited  to  the  range  1 
to  °°  ,  with  the  region  m  <  1  being  outside  of  the  area  where  valid  solutions 
can  be  proven  to  exist,  and  also  note  that  the  root  loci  curves  are  not  plotted 
in  the  plane  commonly  used  for  root  locus  plots  in  linear  control  theory. 

The  selection  of  the  root  to  be  used  in  any  given  case  is  determined  by  the 

criterion  that  the  one  yielding  the  minimum  numerical  value  of  e  +  (e  ,  ,) 

“  n  n  +  re 

is  proper,  and  this  statement  may  be  shown  equivalent  to  the  stipulation: 

Use  the  numerically  larger  value  of  A  e  if 

c 


(en+l ) 


0  < 


nc 


<  (m  -  I  ) 


(  2  : 15) 


otherwise,  use  the  smaller. 

It  is  not  known  whether  the  focal  nature  of  the  rays  revealed  in  Figure 
2:3  is  significant,  nor  if  the  level  line  passage  through  this  point  is  theo¬ 
retically  meaningful;  however,  all  numerical  work  conducted  to  date  seems 
to  indicate  that  the  m  -  9  value  and  its  associated  2  to  1  gain  switch  provides 
consistently  excellent  results. 

It  is  readily  demonstrated  th^  the  level  line  of  Figure  2:3  is  independent 

of  gain  and  of  (e  .  .  This  is  accomplished  as  follows.  We  first  recall 

n  +  1  nc 

equation  2:14, 


4  = 


I  +  m  4-  2Vnr 


(en+l  ) 


nc 


and  find yj_  at  the  limit,  m  =  1  (which  requires  use  of  the  minus  root)  ,  so 

that  we  get,  in  general  at  m  =  1,  letting  be  the  function  value  along  the 
level  line. 


4 


( 2 ;  16  ) 
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This  same  value  of  is  obtained  for  m  =  9,  since  for  m  =  9  (accepting  the 
high  gain  root),  regardless  of  the  value  of  K, 


10  -  2a/9^ 
>nc 


4 


<®n+l  ’ 


nc 


which  completes  our  proof.  For  this  value, 


=  -  (3/2)K(en^.|)^^ 


(2:  17) 


To  establish  if  the  bimdle  of  m  rays  always  penetrates  the  same  point,  regard¬ 
less  of  the  K  and  (e^  ^  values,  it  is  necessary  to  show  that  this  point  is 

common  to  all  rays. 

The  general  equation  for  any  ray  is 


I  m  -  2Vm  2(m  -  i  )  (  ••  1 

''-yTTr-,  J 


(2 ;  18) 


if  all  rays  pass  through  the  focal  point  at  A  e^  =  -  (3/2)K(e^  ^  l^nc’ 
substitution  of  this  value  of  A  should  provide  that  g  =  ,  and  this  is  fCHUid 

to  be  the  case;  therefore,  in  general,  all  rays  pass  through  the  point 


'  •  ■  -  <5/2)K<e«i 

nc 


which  is  the  hi^  gain  root  for  m  =  9,  which  defines  the  two*fto^one  gain 
switching  condition.  This  attematively  proves  that  the  m  =  1  ray  (or  level 
line)  passes  throu^  the  same  point  since  it  is  a  member  of  the  family  defined 


by  2:18. 


Keeping  in  mind  the  relative  locations  of  the  root  loci  curves  and;  the  unrestrained 

solution  value  for  Ae  (at  the  discontinuity  at  the  curve  l/(e  .)  ),  we  see  thtf 
c  n  +  rc 

X  serves  to  ke^  both  the  hi^  and  low  gain  roots  away  from  the  unrestrained 

solution  with  the  hig^  gain  case  representing  a  greater  gain  than  called  for  by 

the  unrestrained  case  and  with  the  low  gain  ctuse  calling  for  a  lesser  gain  than 
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represented  by  the  unrestrained  case.  The  separation  of  the  root  loci  curves 
from  the  unrestrained  solution  is  a  measure  of  the  undetermined  p^ameter  C 
(i.e. ,  C  and  X  are  directly  related)  and  the  sign  of  C  is  seen  to  be  established 
by  the  root  selection  process.  Thus,  C  is  seen  to  be  the  equivalent  of  the  slack 
variable  of  linear  programming,  inversely  the  multiplier  X  serves  the  same 
pui^mse  as  C,  since  C  and  X  are  not  independent. 

if  we  presiune  a  perfect  system  such  that  a  given  command  A  e  ,  could  be 

c 

perfectly  fulfilled,  the  unrestrained  solution  would  represent  a  boundary  of 

neutral  stability,  with  each  command  Ae  resulting  in  overshoot  in  error  act 

c 

the  (n  -f  2)  point  equal  to  the  error  at  (n).  In  an  actual  system,  this  boundary 
is  reached  by  increasing  K  of  a  linear  error  and  error  rate  system  until 
neutral  stability  is  achieved.  Assuming  that  this  K  is  shown  (and  it  can  be 
found  by  standard  linear  methodology),  then  the  damping  of  an  optimum  con¬ 
trol  system  using  this  same  value  of  K  should  be  a  function  only  of  X  or  its 
equivalent  m.  It  would  appear,  therefore,  that  a  stability  criterion  can  be 
devised  for  the  optimum  micro  control  logic  by  extension  of  linear  theory, 
with  the  recognition  that  the  high  gain  roots  wilt  lie  beyond  the  unrestrained 
stability  boundary  in  all  cases.  At  this  point  it  appears  (intuitively)  that  the 
m  =  9  root  has  some  special  significance;  however  there  is  at  present  no 
abstract  proof  to  substantiate  the  intuitive  reaction.  Numerical  data  are 
available  which  establish  only  that  m  -  9  is  an  excellent  average  or  basic 
value  for  the  switching  logic. 

A  review  of  macro-micro  control  theory  is  pertinent,  since  this  theory 
provides  variations  not  considered  above. 

It  can  be  demonstrated  that  the  introduction  of  a  response  factor  relating 

a  macro  command  to  a  micro  (or  autopilot)  function  is  equivalent  to  the 

inclusion  of  the  term  e  in  the  relation  for  (e  .J  ,  i.e. , 

n  n  +  1  nc 

.2 

(en+l  =  ®n  ^  ^  ®n  (  2  :  1 9  ) 


and  that  the  response  factor  "r”  is  given  as  r  =  (e  /0  )  At/2  where  6  is  the 

n  n 

micro  control  rate  and  e  is  the  error  acceleration  of  the  macro  controlled 
quantity. 

When  r  is  included  in  the  analysis,  we  get  that 
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m 


(  2  :  20) 


=  \rAt^/2 

K  =  2/krA1-^  (2  !  2  I  ) 

Jt  is  clear  that  for  a  given  value  of  X,  r  must  be  limited  to  those  values 
which  produce  a  value  of  m  greyer  than  one. 

In  a  macro  control  case,  the  generation  of  a  command  is  removed  from  the 
macro  control  quantity  level  by  at  least  one  step  and,  therefore,  e  cannot 
be  instantaneously  changed.  Thus,  for  fixed  X,  there  can  exist  a  variable 
switching  ratio.  Under  direct  control  (micro  control),  e^  can  be  changed 

instantaneously  and,  therefore,  cannot  be  used  to  alter  gain  switching 
characteristics. 

It  has  been  found  that  in  a  sample  data  system,  the  prediction  time  must 

be  greater  than  the  sample  time  (preferably  at  least  5  times  as  great),  so 

that  the  system  stability  is  associated  with  the  ratio  of  conditions  e}q)ected 

at  two  different  times  in  the  future,  i.  e. ,  at  t  +  At  and  at  A  t  ,  whei’e 

p  .s’ 

"  A  tp  =  prediction  time  and  A  t^  =  sample  time.  In  an  analog  system  the 

ratio  of  prediction  to  sample  time  is  infinite  so  that  such  systems  display 
better  damping  than  sample  data  systems. 

Aside  from  the  effects  of  the  ratio  At  / At  ,  it  aK>ears  that  K  and  m  may 

P  s 

be  independently  established  and  that  K  should  be  equal  to  or  greater  than 
the  value  leading  to  neutral  stability  of  an  average  linear  system. 

This  completes  our  basic  though  brief  analytic  examination  and  development 
of  the  DOLIAC  logic  concept,  and  we  can  now  list  a  few  pertinent  conclusions. 

(1)  The  logic  can  properly  be  called  "optimum"  in  a 
mathematical  sense  since  it  was  devised  using  a 
mathematically  proper  optimiun  definition. 

(2)  Although  we  have  not  carried  forth  the  development 
here,  it  is  shown  in  the  book,  "Principles  Underling 
Systems  Engineering"  Pitman  Publishing  Corporation, 

1962,  that  the  sufficient  conditons  for  the  existence 

of  a  solution  are  satisfied  by  the  DOLIAC  logic.  Thus, 
both  the  necessary  and  sufficient  conditions  are  ful¬ 
filled. 
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(3)  The  logic  is  practical,  since  it  does  not  require  a 
complex  network  of  sensors  for  its  operation,  nor 
more  than  an  elemental  computing  system  for  its 
implementation.  Moreover,  it  requires  a  minimum 
amount  of  input  data  and,  therefore,  may  be  referred 
to  as  a  "limited  information  system." 

(4)  Because  the  Lagrange  multiplier  behaves  like  a 
slack  variable,  one  must  still  face  the  problem 
of  proper  gain  selection. 

(5)  The  theory  is  nonlinear  because  of  its  optimiun 
switching  criterion,  and,  therefore,  stability 
studies  have,  up  to  this  time,  had  to  be  carried 
forth  using  full  digital  and  analog  simulation 
methodology.  Studies  carried  forth  at  Cornell 
Aero  Laboratories  and  at  DODCO  have  demon¬ 
strated  that  linear  stability'  analysis  techniques 
do  not  ^ply.  Indeed,  the  logic  is  stable  in 
regions  where  conventional  linear  theory  indi¬ 
cates  it  to  be  unstable. 

(6)  The  theory  presented  represents  only  a  first  step 
beyond  linear  theory  and,  therefore,  future  improve¬ 
ments  are  possible;  however,  the  direction  which 
should  be  followed  is  not  defined  at  this  time. 

In  the  next  and  final  section  of  this  paper,  we  shall  discuss  the  more  pragmatic 
nature  of  the  DOLLAC  logic,  and  the  nature  of  the  results  obtained  from  extensive 
application  studies  of  the  DOLLAC  logic  in  its  micro  and  macro-micro  forms. 

3.)  RESULTS  AND  CONCLUSIONS 

Over  the  course  of  the  past  four  years  the  group  at  DC^CO,  as  well  as  other 
investigators,  have  extensively  analyzed  the  nature  of  the  DOLLAC  logic  from 
various  standpoints,  and  these  investigations  have  shown  that  the  logic  is 
inherently  "laz}^'  in  that  it  does  not  switch  gain  unless  drift  occurs  or  external 
disturbance  and/ or  abrupt  changes  in  command  are  introduce.  Thus,  for  long 
periods  of  time  the  system  lies  partially  dormant  operating  principally  at  one 
gain  setting;  however,  even  in  such  regulatory  phases,  as  drift  develops,  short 
bursts  of  hi^  gain  operation  ocettr  in  accord  with  the  switching  logic  require¬ 
ments.  Under  idealized  circumstances,  therefore,  it  is  possible  to  achieve 
system  states  which  call  for  almost  completely  linear  operation  of  the  DOLlAC 
logic,  and  studies  based  wdiolly  on  such  circumstances  can  produce  extremely 
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misleading  results,  since  under  these  circumstances  one  can  resuiiiy  find  a 
describing  function  A^ich  apparently  represents  the  system,  but  vdiich  has, 
in  fact,  no  meaning  outside  of  the  simplified  set  of  operating  circumstances 
assumed  for'liuialysis.  Similarly,  if  the  problem  given  to  the  system  is 
idealized,  one  can  be  led  to  the  false  conclusion  that  stability  is  predictable 
using  the  Evans  root  locus  technique,  but  again  the  application  is  quite 
meaningless  (in  a  general  sense)  since  the  switching  characteristics  depend 
on  what  occurs  ta  any  given  time  and,  therefore,  the  poles  and  zeros  of  any 
assumed  linear  equivalent  system  move  about  as  functions  of  time  (and 
incidentally  as  functions  of  the  particular  error  state). 

As  mentioned  in  the  introduction,  the  fact  does  remain  that  the  DOLIAC 
logic  will  assume  a  quasi-linear  state  whenever  possible,  and  this  indicates 
that  simple  error  and  error  rate  logic  can  indeed  be  optimum  under  some 
circumstances.  The  difficulty  here,  however,  is  that  these  circumstances 
can  never  continuously  be  obtained  in  any  practical  application,  and  the 
main  reason  for  using  optimunA  or  adaptive  logic  is  to  account  for  real 
rather  than  purely  idealized  situations.  Thus,  the  superiority  of  the 
DOLIAC  logic  over  linear  logic,  per  se,  resides  only  in  the  fact  that  the 
DOLIAC  logic,  because  of  its  switching  law,  can  instantaneously  and 
"logically"  shift  g^  vdienever  it  needs  to. 

At  this  point  it  is  of  interest  to  note  that  if  we  sou^  to  minimize  the 

quantity  (e  )^  as  a  criterion  for  optimum  performance,  subject  to  the 

same  type  of  inequality  constraint  introduced  in  Section  2,  then,  by  simple 
manipulation,  we  at  once  obtain  a  command  logic  of  the  form 


which  is  readily  verified  by  the  reader.  This  is  the  standard  linear  error 
and  error  rate  control  equation.  Now,  since  this  equation  is  derivable 
from  an  error  squared  criterion  (recftll  that  the  error  squared  criterion 
tends  to  diminish  emphasis  on  regulation  as  contrasted  to  control),  we  see 
that  another  way  to  separate  the  DOLIAC  and  linear  logics  is  purely  on  a 
criterion  basis.  Thus,  if  we  tamper  with  the  criterion,  we  can  eliminate  the 
switching  logic  and  end  up  with  what  is  still  matltematically  an  optimum 
solution,  but  not  to  the  problem  we  are  obliged  to  solve.  We  conclude  that 
linear  theory,  therefore,  seeks  to  minimize  error  squared  and  if  this  is 
desirable,  then  linear  theory  should  be  employed. 


It  is  now  pertinent  to  ask,  how  is  the  DOUAC  logic  related  to  other 
switching  logics,  of  A^lch  several  have  been  prc^osed?  The  answer  to 
this  is  simple.  The  DOUAC  switching  logic  is  a  result  of  appl3dng  a 
specific  criterion  and  a  specific  root  selection  process  both  of  which 
axe  entirely  independent  of  the  specific  control  problem.  Thus,  this 
logic  results  from  definition  and  analysis  rather  than  from  empiricism 
and  extrapolation.  In  other  words,  the  DOUAC  logic  has  been  designed 
on  the  basis  of  purely  fundamental  considerations,  and  as  such  is  related 
to  conventional  control  logic  only  after  the  fact.  Its  resemblance  to  con¬ 
ventional  logic  is  conicidental— not  purposeful,  and  because  this  ,  is  true, 
the  DOUAC  developments  serve  to  justify  the  efforts  in  classical  control 
theory. 

To  the  control  engineer,  one  of  the  major  theoretical  drawbacks  of  the 
DOUAC  logic  resides  in  the  fact  that  its  development  provides  no  clue 
as  to  its  stability  characteristics.  This  is  a  theoretical  problem  re¬ 
quiring  fiuther  study;  however,  the  ^plication  of  linear  control  logic 
to  non-linear  dynamic  systems  is  equally  displeasing  in  a  theoretical 
sense,  and  since  no  physical  system  is  really  linear,  the  pr^ical  dif¬ 
ferences  in  the  application  of  the  DOUAC  logic  and  linear  theory  do  not 
constitute  a  problem  of  real  magnitude.  As  a  matter  of  fact,  a  simple 
rule  of  thumb  exists  which,  although  conservative,  makes  practical  appli¬ 
cation  of  the  DOUAC  logic  rather  simple.  The  rule  is  "If  one  analyzes  the 
DOUAC  logic  as  a  constant  gain  linear  system  and  it  is  stable  for  the  low 
gain,  then  the  full  logic  is  also  stable  although  a  linear  study  will  show 
that  constant  high  gain  operation  wculd  be  unstable."  .This  provides  a 
simple  method  of  selecting  the  gain  constant  K.  Additionally,  the  ratio 
of  prediction  to  sample  times  should  be  five  or  more,  A  system  designed 
on  the  basis  of  these  considerations  will  perform  more  effectively  than 
any  stable  linear  logic;  however,  there  are  some  limitations  to  observe. 

The  switching  logic  and  the  location  of  the  switching  lines  in  an  errors 
error  rate  phase  plane  are  entirely  dependent  on  the  measured  error  to 
error  rate  ratio  and  should  this  ratio  be  improperly  determined,  the 
systems  performance  suffers.  The  most  critical  sensor  is  the  one  which 
is  used  to  find  the  highest  level  error  derivatives,  and  as  a  rule  of  thumb, 
the  natural  frequency  of  this  sensor  should  exceed  the  system  normal 
response  frequency  by  an  order  of  magnitude.  Because  of  this  restriction 
the  DOUAC  logic  should  not  be  applied  to  the  control  of  systems  wherein 
reasonably  accurate  measurements  of  error  rate  are  not  possible.  As  a 
matter  of  fact,  basic  error  and  error  rate  linear  systems  exhibit  this 
same  characteristic  and,  frequently,  poor  sensor  information  may  be 
worse  than  hone.  The  sensor  probleni  is  not  one  of  resolution  but  rather 
of  phase  lag  since  the  lo^c  can  readily  handle  consistent  residual  errors. 
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but  phase  lags  cf  anywhere  near  90°  cause  trouble  unless  the  prediction  of 
time  At  and  the  basic  gain  factor  K  are  reduced,  with  a  reduction  of  both 

of  them  leading  to  poorer  command  following  ability. 


Jt  should  now  be  of  interest  to  briefly  review  the  applications  of  the 
DOLIAC  logic  vdxich  have  been  investigated  to  date  and  to  note  the 
successes  of  the  logic  together  with  a  significant  failure. 

The  first  application  was  to  a  simple  micro  control  problem  (auto  pilot 
logic)  involving  a  prefilter  on  conimand  inputs.  The  analysis  was  by  means 
of  digital  simulatibn  of  control  of  a  non-linear  vehicle  throughout  a  com¬ 
plete  environmental  spectrum  including  the  subsonic,  transonic  and  super¬ 
sonic  regions  both  at  hig^  and  low  altitudes  and  subject  to  various  classes 
of  command  input.  Both  perfect  and  imperfect  sensors  were  examined. 

The  results  were  extremely  satisfactory  and  superior  to  those  obtained  by 
linear  error  and  error  rate  comparison  systems,  and  also  superior  to  other 
types  of  gain  switching  logics  which  were  simulated.  The  results  obtained 
were  subsequently  checked  at  CAL  using  analog  simulation  of  parts  of  the 
environmental  pattern;  duplication  of  conclusions  was  obtained.  Additionally, 
aut(^ilot  operation  was  checked  digitally  without  a  prefilter  and  satisfactory 
results  were  again  obtained. 

The  second  application  involved  a  macro-micro  control  problem,  namely,  the 
control  of  skin  temperature  of  a  re-entry  vehicle  by  means  of  altering  the 
flight  path  through  a  directly  coupled  autopilot.  The  basic  error  signal  in 
this  case  being  the  difference  in  desired  and  sensed  skin  temperature.  A 
sensed  response  factor  "r"  was  used  in  the  logic.  Results  were  entirely 
satisfactory. 

The  third  application  was  an  ^empt  to  control  ICBM  type  flexible  missiles 
for  which  accurate  measurements  of  pitch  rate  by  even  complex  sensor  net¬ 
works  were  not  possible.  Results  were  unsatisfactory  shice  satisfactory 
error  rate  data  could  not  be  provided  to  the  logic,  and  less  sophisticated 
control  configurations  proved  svqwrior  imder  simulation. 

The  fourth  application  was  to  combine  longitudinal,  lateral  range  and 
skin  temperature  control  cf  a  re-entry  vehicle.  An  entirely  satisfactory 
unified  descent  control  system  was  evolved  capable  of  handling,  adaptively, 
and  environmental  pattern  encountered.  Error  display  information  for  pilot 
control  was  also  generated  by  the  logic. 

The  fifth  applicatibn  was  to  automatic  landing,  descent  and  flare  control, 
and  an  entirely  satisfactory  macro-micro  control  logic  was  synthesized. 
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The  sixth  application  was  to  an  automatic  trim  control  system  designed  to 
operate  in  parallel  with  a  human  pilot,  and  study  of  the  interaction  of 
the  control  logic  and  the  pilot  is  now  being  carried  forth  at  ASD,  A  human 
pilot  is  included  in  the  ASD  simulation* 

The  seventh  application  has  been  to  macro-micro  control  of  explicit  point 
and  explicit  orbit  guided  trajectories,  again  with  entirely  satisfactory 
results* 

The  eighth  application  was  to  quasi-optimum  performance  guidance  and  con¬ 
trol  of  spaceplane  type  vehicles  using  limited  sensors*  Insufficient  work 
has  been  conducted  to  fully  evaluate  this  system;  however,  data  obtained  to 
date  are  favorable* 

The  ninth  application  was  to  the  problem  of  macro-micro  control  of  orbital 
docking  maneuvers,  and  the  results  of  the  extensive  simulation  study  of 
this  system  have  been  extremely  satisfactory. 

Thus,  so  far,  we  have  a  batting  average  of  eight  successes  in  nine  appli¬ 
cation  studies  with  one  failure,  with  the  reason  for  the  failure  being 
quite  clear* 

It  should  be  recognized  that  because  we  have  so  far  been  unable  to  find 
an  entirely  proper  stability  criterion  (the  rule  of  thumb  presented 
earlier  is  known  to  be  too  conservative),  that  the  study  of  the  DOLIAC 
logic  and  its  applications  have  been  largely  pragmatic  in  nature  and  that 
the  stipulations  presented  here  are  in  no  way  based  on  abstraction  or 
idealization,  but  rather  upon  the  very  practical  criteria:  ”Does  it  work?” 
and  ”H6w  does  it  work  compared  to  other  systems?”  The  answers  to  these 
questions  have  been  ”Yes,  it  does  work  in  all  cases  where  the  theory  makes 
sense”  and  ”on  a  comparative  basis  it  seems  to  work  better  than  other  sys¬ 
tems  we  have  compared  it  to*”  By  no  means  have  we  compared  all  possible 
systems,  nor  do  we  in  any  sense  feel  that  the  present  logic  is  an  ultimate 
form,  rather  we  feel  that  it  represents  a  first  step  forward,  that  is,  it 
doubtlessly  can  be  improved  upon,  that  it  does  call  for  more  stucfy  and 
that  it  provides  an  affirmative  answer  to  the  question,  ”Is  optimum  synthe¬ 
sis  practical?”  Space  has  not  permitted  more  than  a  summary  of  the  several 
thousands  of  pages  of  analyses  and  data  which  have  resulted  frorh  study  of 
the  DOLIAC  logic*  However,  the  basic  theoretical  elements  are  contained 
in  this  paper  and  listed  below  are  a  group  of  selected  references  which  may 
be  consulted  for  further  information. 
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Finally,  we  want  to  note  that  the  present  logic  is  the  result  of  evolution 
and  that  the  early  DODCO  attempts  at  optimum  adaptive  synthesis  reported 
on  in  reference  3  were  vastly  different  from  those  discussed  in  this  paper. 
Also,  we  want  to  express  our  thanks  to  A$D  and  to  C^t  Raymond  Rath  and 
Mr.  P.  C.  Gregory  for  their  initial  interest  in  this  effort  and  for  their 
continued  support  of  a  rather  complex  area  of  research  which  has  had  its 
moments  of  extreme  frustration. 
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THE  APPROXIMATE  REALIZATION  OF  OPTIMUM 
CONTROL  SYSTEMS 
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P.J.  Alexandro,  Jr. 

ABSTRACT 

Ihe  paper  describes  two  ways  of  approximate  realization 
of  Pontryagln  and  Bellman  theories.  The  more  sophisticated  method 
involves  combining  the  features  of  both  theories  and  use  of  both 
digital  and  analog  feedback.  The  less  sophisticated  but  more  eco¬ 
nomical  method  en^loys  simple  analog  nonlinear  elements.  Both 
methods  are  applicable  to  high  order  systems  with  random  as  well  as 
deterministic  inputs.  A  comparison  of  the  limitations  and  similar- 
ities  of  the  two  methods  are  given. 

Both  methods  display  bang-bang  characteristics  for  large 
inputs,  but  linear  stable  characteristics  when  the  error  signal  is 
low.  A  reserve  control  force  is  provided  (in  both  methods)  to 
counteract  random  distrubances,  euid  minor  variations  in  plant  trsins- 
fer  function. 

The  more  economical  method  is  applied  to  the  positioning 
and  stabilization  of  an  inertial  platform  which  has  a  fourth  order 
transfer  function  and  appreciable  load  disturbances.  Analog  Btvdlea 
show  that  the  approximate  optimum  control  not  only  out-performs  lin¬ 
ear  controls  in  every  way  by  a  wide  margin,  but  is  also  moch  simpler 
to  implement  and  requires  less  weight. 
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INTRODUCTION 


Tvo  types  of  difficulties  are  encountered  In  applying 
optimum  control  theories: 

1.  Physical  difficulties 

a.  the  existence  of  random  noise  and  disturbance. 

b.  the  lack  of  knowledge  and/or  the  change  In  plant  dynamics. 

2.  Computational  difficulties 

a.  the  two  point  boundary  value  problem  in  maximum  principle. 

b.  the  requirement  of  large  storage  capacity  in  dynamic  pro¬ 
gramming. 

The  present  paper  deals  with  methods  of  overcoming  these 
difficulties.  For  the  convenience  of  utilizing  an  earlier  report 
without  costly  revision,  the  paper  Is  made  up  of  two  Integral  parts 
with  Independent  equation  and  figure  numbers.  Part  I  (by  Chang)  Is 
on  the  general  philosophy  and  method.  Penrt  II  (by  Alexandro  and 
Chang)  Is  on  a  special  application  In  which  the  general  philosophy 
of  Part  I  Is  realized  In  a  very  sliiq>le  and  practical  manner.  It 
should  be  noted  that  while  the  special  problem  dealt  with  In  Ps^  II 
Is  that  of  optimum  control  of  an  Inertial  platfom,  the  method  Is 
applicable  to  any  system  with  no  more  than  two  dominant  lagging  time 
constants. 


271 


PART  I 


GENHtAL  PHILOSOPHy  AUD  METHOD 


SystanB  vlth  Random  Wolse  and  DlBtur^)ance 

A  system  vltb  razidom  disturbances  can  be  descrtbed  by  the 
set  of  equations 


where  ^  u,  azid  v  are  the  state  vector,  control  vector,  and  distur¬ 
bance  vector  respectively,  and  f  is  a  vector  function.  Generally, 

X  and  f  have  the  same  dimensions,  while  the  dimensions  of  u  and  v 
are  different.  The  function  f  has  the  proper  continuity  properties 
so  that  given  x(tQ),  and  u(t),  v(t)  for  all  t  >  t  ,  x(t)  is  uniquely 
determined. 

The  control  vector  u  can  be  selected  at  will  within  given 
constraints  either  as  a  function  of  x  or  as  a  function  of  t.  Fol¬ 
lowing  Bellman,  we  shall  refer  to  such  a  selection  as  a  control 
policy.  In  a  deterministic  system,  v(t)  =  0.  Given  the  control 
policy,  and  x(‘^o)  subsequent  x(t)  is  completely  determined, 
therefore  we  can  use  x(t}  to  describe  the  system.  In  a  system  with 
random  disturbances,  however,  v(t)  is  not  zero  nor  predictable. 

Given  the  control  policy  and  x(  to)  >  "the  subsequent  x(t)  is  unknown. 
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Ihe  only  predictable  aspect  of  the  system  is  then  a  probabilistic 
one;  Let  At  represent  a  small  volume  element  in  state  space  about 
then  the  probability  of  finding  the  representative  point  x(t) 
of  the  system  in  At  is  proportlonsil  to  the  volxme  of  At: 

p(x(t)  €  At)  =  p(xi  ,  t)  AT  (2) 

!rhe  probability  density  p  is  a  function  of  two  variables:  the 
point  ^  under  consideration  and  time  t.  The  uncertain  state  of 
the  system  is  represented  by  p(3^  ,  t).  If  as  t  increases,  p  con¬ 
verges  towards  a  small  neighborhood  near  the  desired  point  ^  in 
state  space,  the  system  is  a  convergent  and  satisfactory  system. 

If  p  spreads  out  the  system  is  not  satisfactory.  The  speed  and 
compactness  of  the  convergence  of  p  towards  ^  are  measures  of  the 
merit  of  the  system. 

Partial  DifferenticJ>  Eduation  of  p 

In  most  control  systems,  the  randcsn  disturbances  sire  small 
compared  to  the  control  forces,  and  (1)  can  be  written  as 


X  =  f(x,u,0,t) 


(3) 


Given  the  control  policy,  u  is  a  function  of  x  and  t,  and  f(x,u,0^t) 
can  be  written  u  F(x,  t). 
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Eq.  (3)  can  be  rewritten  as 


X  =  F(x,  t)  +  Y  vj  (4) 

L  3vi 

i 

Bq.  (If)  describes  the  system  with  a  slngp,e  sample  of  v. 
Considering  all  possible  v,  the  variation  of  probability  density  p 
with  t  at  any  particular  location  can  "be  described  by 


where  (3p/3t)o  gives  the  part  due  to  mean  motion  F,  and  (dp/9t)v 
gives  the  part  due  to  random  disturbance  v. 

The  probability  density  can  be  thought  of  as  the  substance 
or  mass  of  a  fluid.  Hie  conservation  equation  in  hydrodynamics  gives 

The  effect  of  the  random  distiurbances  is  to  cause  a  diver¬ 
sion  of  the  system  frcxn  its  e^qpected  path^  The  sum  total  of  these 
diversions  is  a  "diffusion”  of  the  probable  states  of  the  system. 

We  note  that  in  Eq.  (3)  v  is  a  linear  additive  term.  If 
a  certain  v(t)  occurs  for  an  interval  6t,  the  representative  point 
X  is  displaced  by  an  amount  6x  which  is  independent  of  x.  We  can 
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therefore  imagine  that  the  entire  distribution  pattern  p(x,  t)  is 
shifted  by  6x.  The  new  p  at  Xj  is  therefore  identical  with  the  un 
disturbed  p  at  x^  -  6x.  The  change  in  p  Is 

6p  =  p(j^-6x,  t)  -  p(x^,  t) 


a^p 


i  j 


axidxj 


Taking  ensemble  average  of  (7)  gives 

^  J 

The  ensemble  averages  of  the  odd  order  terms  vanish  because 
V  and  -V  are  equally  likely,  and  the  fourth  and  higher  order  terms 
are  negligible. 

In  control  systems,  the  disturbances  generally  have  much 
wider  bandwidths  than  that  of  the  system.  In  terms  of  time,  the 
correlation  times  of  v^  eire  much  shorter  than  the  system  time  con¬ 
stants.  We  can  select  a  6t  which  is  much  shorter  than  the  system 
time  constants  but  longer  than  the  correlation  time  of  v±,  and  cal¬ 
culate  the  ensemble  averages  { 6x1 )  f  6x jj  occasioned  by  v  in  time  6t. 
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Let  the  Initial  time  be  denoted  t^ .  Eq.  (3)  gives 


6x(t)  -  ^ 

(9)“ 

t 

1 

t 

> 

6x(t)  = 

6x(t)  dx  =  ^ 
w  1 

af  1 

'-5^ 

L  ^ 

Vi(t)  dT 

(10) 

The  partial  dertvatlves  can  be  left  outside  the  integral  sign  be¬ 
cause  they  do  not  change  appreciably  in  any  interval  which  is  much 
shorter  than  the  system  time  constants.  It  follows  from  Eqs.  (9) 
and  (10) 


dt 


[(6x£)(6xj)]  =  (6xi'(6xj)  +  (6xj)(6xi) 


af.  af. 


k  T  ^^1 


[Vk(t)  Vi(t)  +  Vk(T)  V2^(t)]  dT 


Taking  ensemble  average  of  the  above  eqxiation  gives 


af^  df 


J 


avv  Sv,  I 

k  1  *  1  J 


[♦kl(T-t)  +  ♦kl(*-'r)l 


(11) 


where  is  the  correlation  function  of  vj^  and  v^.  Substituting 
X  for  t-T  in  Eq.  (11)  and  integrating  the  result  from  ti  to  ti+6t 
gives 


i 
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_  Sf,  3f, 

(8xiK6x,)  .^2. 


tx+6t  t-ti 


j 

^  ^  ti  o 


(^kl("^)  ■•■  ♦kl(^)l 


yy  ilL  ifj 

ty  j 


St 


[•kl(-k)  t  •kl(8)l  («t-S)  dt  (12) 


Since  the  correlation  time  la  very  short, 

6t 


[♦"^(-^0  •'i  ♦i,i(^)]  (6t-X)  dX 


'kl 


6t 


00 


6t 


♦ui(X)  dX  =  6t  *u.(0) 


-00 


(13) 


^ere  #kl(j<u)  is  the  cross- spectral  density  of  vjt  and  vi.  Eq-.  (12) 
becomes 


(6xi)(6xj)  =  Dij  6t 
The  factors  D^j  are  defined  as 


(lU) 
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♦lcl(o) 


(15) 


k  1 


dvn 


Substituting  (14)  into  (6)  gives 


Therefore 


6p 


axidxj 


DiJ 


6t 


..afP- 


(16) 


Finally,  by  substituting  (6)  and  (l6)  into  (5)  we  obtain 


l£. 

at 


-V  (pFl^l^^^Dtj 
i  J 


9xj^&Xj 


(17) 


Bq.  (17)  is  a  partial  differential  equation  which  gives  the  varia¬ 
tion  of  p  with  time.  Its  general  solution  is  quite  beyond  our 
present  stage  of  development  in  mathematics.  However,  by  examining 
a  few  special  cases,  we  cw  gain  some  insight  on  the  effects  of 
random  disturbances  on  systems  of  different  desi^s. 
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One  Dimensional  Solutions 


Consider  the  sajnple  case  with  a  state  v€U"iable  x  of  only  one 
dimension.  Equation  (l?)  is  reduced  to 

Exajnple  1.  The  function  F  is  given  by 

F  =  -ax 

and  D  are  constants.  In  steady  state  Eq.  (18)  becomes 

Integrating  (19)  gives 


**"  ^  (^0) 

where  Cx  is  a  constant  independent  of  x.  Note  that  in  order  to  satis¬ 
fy  the  equation 

00 

*> 

pdx  =  1  (21) 

C 

-00 

both  the  product  px  and  must  vanish  at  infinity.  Bierefore  Ci  =  0, 
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and  (20)  becomes 


JL  =  0 

p  dx  D 

Integrating  the  above  expression  gives 


P 


C2  e 


-ax^/D 


(22) 


(23) 


where  C2  is  a  constemt  and  can  be  determined  from  (21)  ,  Eq.  (23)  re¬ 
presents  a  packet  of  f.m.s.  distance  /0/2a  from  the  origin.  The  sys¬ 
tem  converges  eventually  into  this  packet. 


Example  2, 


F(x)  =  0  for  X  >  0 


There  is  no  restoring  force  on  the  positive  side  of  x.  It  will  be 
shown  in  general  that  there  is  no  steady  state  solution  and  the  pro¬ 
bability  density  flows  in  the  positive  x  direction  until  all  is  ex¬ 
hausted. 

The  total  probability  of  the  system  above  some  positive  value 
of  X,  say  b  is  given  by 


p(x  >  b) 


00 

p(x,t)  dx 
b 


As  F  =  0,  Eq,  (18)  gives 
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(24) 


Since  dp/dt  represents  the  rate  of  increase  of  p(x  >  b) ^  conservation 
law  requires  that  the  expression  on  the  right  hand  side  of  Eq.  (24) 
represents  the  rate  of  flow  of  the  probability  density  passing  the 
point  X  =  b.  As  b  is  arbitrarily  selected,  this  relation  holds  for 
all  positive  values  of  x: 

Rate  (a)  .  -  ^  (-1^)  (25) 

The  integrated  rate  of  flow  is 


00  00 

P 

Rate  (x)  dx  - - ^  dx  =  [p(o)  -  p(oo)]  (26) 

J  V. 

o  o 

In  order  to  satisfy  (21),  p(cd)  =  0.  While  (26)  does  not  rule  out 
treuisient  flow  in  the  -x  direction  at  some  particular  location,  it 
does  prove  that  on  the  whole  the  probability  density  moves  out  in  the 
tx  direction  until  p(0)  =  0. 

Practical  Implications  -  Measiare  to  Counteract  Disturbances 

While  the  above  calcxilations  are  for  a  one-dimensional  prob^ 
lem  about  x  =  0,  the  situation  is  basically  the  same  in  a  multi¬ 
dimensional  problem  about  the  expected  or  undisturbed  path.  In  a 
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bang-bang  control  system,  the  switching  surfaces  can  be  followed  only 
by  exerting  maximum  available  control  force.  Once  the  representative 
point  is  sent  beyond  the  switching  surface  by  the.  disturbances,  there 
is  no  available  control  force  to  restore  it  to  the  expected  path. 

The  situation  is  quite  similar  to  that  of  Example  2.  The  probability 
is  overwhelming  that  the  system  does  not  follow  the  expected  path  or 
even  stay  close  to  it. 

A  corrective  measure  is  siaggested  by  Example  1,  If  the  con¬ 
troller  is  so  designed  that  a  large  part  but  not  all  the  control 
force  is  used  to  follow  the  expected  trajectory,  a  linear  corrective 
region  can  be  set  up  about  the  latter.  The  required  linear  range  is 
no  more  than  /B/a.  The  probability  density  p  is  then  focussed  into 
a  small  packet  about  the  expected  path  or  x( t) .  Unless  the  distur¬ 
bances  are  extremely  severe,  the  required  margin  in  control  force  is 
comparatively  small.  It  is  a  small  price  to  pay  to  gain  surer  control. 

Measure  to  Counteract  Uncertainties  in  Plant  Dynamics 

There  ore  many  papers  published  on  the  subject  of  adaptive 
control,  model  feedback  systems  etc.,  and  our  discussion  on  this  sub¬ 
ject  will  be  brief.  Two  methods  stand  out  as  effective  measures  to 
counteract  plant  uncertainties: 

1.  Estimating  the  plant  dynamics  from  past  response,  and  using 
the  latest  estimate  for  the  computation  of  optimum  trajectory. 
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2.  Employing  minor-loop  feedback  to  force  the  system  response 
to  a  preselected  model. 

Of  the  two  methods^  (1)  is  usually  slower  and  more  costly  in 
comparison;  but  for  slow-varying  systems  with  a  wide  range  of  varia¬ 
tion;  it  has  the  advantage  of  making  better  or  more  effective  utili¬ 
zation  of  the  control  forces.  For  these  reasons;  (2)  perhaps  should 
be  employed  first;  and  (l)  is  used  in  addition  only  in  case  of 
necessity. 

In  order  to  make  (2)  effective;  the  required  minor-loop 
gain  is  usually  quite  high.  Thus  lead  networks  are  needed  with  high 
noise  as  a  result.  An  alternative  is  to  use  the  directly-measured 
highest  time -derivative  as  the  feedback  signal.  This  has  the  ad¬ 
vantage  of  lead  stabilization  but  without  the  difficulties  resulting 
from  differentiating  instrument  noise. 

Mathematical  Difficulties  and  A  Counter  Measure 

There  are  two  basic  methods  of  optimal  control:^  Bellman's 
dynamic  programming  and  Pontryagin's  maximiam  principle..  Their  mathe¬ 
matical  difficulties;  which  are  mentioned  in  the  introduction;  are 
well  known  and  do  not  need  further  explanation.  The  proposed  counter 
meas\ire  utilizes  features  of  both  theories.  Consider  the  problem 

1.  For  an  introductory  description  of  these  methods  see  Chapters  9 
and  12  of  S.  S,  L.  Chemg;  ’’Synthesis  of  Optimum  Control  Systems/’ 
McGraw  Hill,  New  York,  1961. 
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of  sending  the  system  from  a  fixed  initial  point  to  a  fixed  final 
point  X2  with  a  minimum  of  cost .  The  least  cost  from  each  point  x  in 
the  neighborhood  of  2^  can  be  coinputed  by  dynamic  programming.  The 
constant  cost  surfaces  in  the  neighborhood  of  3^  are  then  stored  in 
the  computer.  By  assuming  an  initial  covariant  vector  an  optimum 
trajectory  can  be  computed  to  go  from  ^  to  some  point  2^  in  the 
neighborhood  of  2^.  With  a  fast  computer,  a  few  of  these  trajectories 
can  be  computed  before  a  decision  needs  to  be  mde.  The  one  with 
least  total  cost  is  then  selected.  The  basic  idea  is  illustrated  in 
Fig,  I-l. 

Because  only  the  cost  vedues  of  points  in  a  small  neighbor¬ 
hood  of  ^  needs  to  be  computed  eind  stored,,  the  required  storage  ca¬ 
pacity  is  usually  not  prohibitive.  Because  the  final  point  of  the 
Pontryagin  trajectory  is  a  finite  neighborhood  rather  than  a  single 
point,  the  trial  process  in  the  selection  of  an  initial  covariant 
vector  is  also  not  overly  difficult.  In  any  case  the  overall  diffi¬ 
culty  can  be  minimized  by  a  sensible  selection  of  the  size  of  the 
neighborhood  about  xg. 

A  ibroposed  Digital  Control  System 

A  combination  digital-analog  controller  which  embodies  all 
the  above  discussed  coiinter  measures  is  illustrated  in  Fig.  1-2. 

Using  the  method  of  Fig.  I-l,  a  nearly  optimum  trajectoiy  is  computed 
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by  the  digital  computer  on  the  basis  of  a  meucimum  control  force  which 
is  no  more  than  a  large  fraction  (say  905^)  of  the  actual  available 

value.  There  are  two  outputs  from  the  computers  the  optimal  con- 

A  A 

trol  force  u  and  the  desired  acceleration  x.  The  actual  control 

force  applied  to  the  plant  is  u  6u,  where  6u  is  the  corrective 

force  to  keep  x  "  ^  small , 

Besides  the  immediate  feedback  signal  x^  the  actual  x  is  al¬ 
so  measured  and  this  information  is  fed  back  to  the  computer.  From 

A 

the  measured  x  and  u  +  6U;  the  computer  estimates  the  system  dynamics 
and  uses  this  information  and  the  measured  x  to  revise  the  optimal 
path  data  u  and  x.  The  parameter  oc  in  Fig.  1-2  is  used  to  indicate 
that  the  system  dynamics  may  change  . 

There  is  another  reason  for  using  the  acceleration  signal 
for  inner  loop  feedback:  In  order  for  such  feedback  to  be  effective, 
the  loop  gain  must  be  high,  and  time  delay  in  measuring  the  feedback 
variable  must  be  kept  at  a  minimum.  Suppose  the  controlled  plant  is 
a  space  vehicle.  Its  acceleration  can  be  measured  directly  and  in- 
stemtly  right  on  the  vehicle,  but  position  and  velocity  have  to  be 
determined  in  roundabout  ways. 

The  general  method  illustrated  in  Fig.  1-2  is  overly  elaborate 
or  too  costly  for  most  applications .  However,  if  the  plant  has  only 
two  dominant  lagging  time  constant  (including  l/p  which  represents 
the  limit  of  T  oo),  and  its  transfer  function  is  known,  the  counter- 
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OPTIMAL  TRAJECTORES. 

'by (DIGITIZED)  MAXIMUM  PRINCIPLE 


Figure  I-l.  Computation  of  Optimal  Trajectory 
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Disturbances 
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Figure  1-2.  Computer-Controlled  Nonlinear  System 


measures  discussed  above  can  be  realized  by  a  very  slsiple  cuid  preictlcal 
controller  using  only  analog  elements.  Die  sisipler  model  vlU  be  de¬ 
scribed  in  BEtrt  II. 


PART  II 

APPROXIMATELY  OPTIMUM  CCfflTROL  OF  AN 
INERTIAL  PLATFORM  WITH  RANDC^l  DISTURBANCES 


In  this  peurt  of  the  paper  ve  will  discuss  a  number  of  tech¬ 
niques  which  can  be  used  to  modify  the  bang-bang  principle  in  order 
to  overcome  some  common  problems  such  as  finite  gain,  random  inputs, 
and  minor  time  constants,  namely: 

1.  Automatic  dual  mode. 

2.  Provision  for  reserve  torque 

3-  Means  for  incorporating  lag  conipensation. 


288 


An  exas^le  vlU  also  be  given  In  vhlch  the  methods 
are  incorporated  Into  an  approximate  bang-bang  system  to  con¬ 
trol  an  Inertial  platform. 

Automatic  Dual  Mode  System 

The  problem  of  positioning  an  inertial  load  vhlch 
may  be  subject  to  large  Inputs  of  displacement  was  considered 
by  McDonald  in  I95O. ~  He  demonstrated  that  the  system  could 
be  made  to  respond  to  a  st^  input  In  the  minimum  time  by 
using  a  relay  controller  which  reverses  the  torque  In  accord¬ 
ance  with  the  sign  of  the  es^ression; 

-  e  +  e  I  e  1  ; 

where  T  =  Maximum  torque  output  , 

J  s  Inertia  of  load  , 

e  =  Error 

TJie  dividing  line  is  shown  in  Fig.  1.  By  controlling  the 
torque  in  this  manner  the  system  will  follow  the  optimum 
trajectory  In  the  phase  plane  and  only  one  torque  reversal  is 
required  in  order  to  bring  the  ou'^ut  to  the  correct  position. 

The  disadvantage  of  the  relay  controller  is  that  due 
to  small  time  delays  present,  a  Usdt  cycle  will  normally 
exist;  with  the  result  that  the  performance  of  the  system  will 
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be  marred  by  the  presence  of  small  oscillations  in  the  output. 
In  addition,  mnan  load  disturbances  will  cause  much  larger  er¬ 
rors  in  a  relay  system  than  they  vould  in  a  linear  system. 

To  remedy  the  defects  of  the  releiy  system,  McDonald 

p 

proposed  a  dual  mode  servo  system  vhich  vould  combine  the  ad¬ 
vantages  of  a  relay  system  end  a  linear  system.  For  large  in¬ 
puts  the  torque  is  controlled  hy  a  relay  so  as  to  follow  the 
optimum  tn^jector^'  in  the  phase  plane.  When  the  magnitude  of 
the  error  is  reduced  below  a  certain  value,  the  relay  is 
switched  out  and  replaced  hy  a  linear  aoplifier.  Thus  optimum 
switching  can  be  eapiayed  for  large  errors,  \rtiile  avoiding 
both  the  limit  cycle  inherent  in  a  relay  system,  and  the  in¬ 
effectiveness  of  the  relay  servo  in  compensating  for  small 
random  load  dlstiurbances . 

It  is  not  necessary  to  switch  between  a  relay  and  a 
linear  aoplifler  in  order  to  obtain  the  desirable  character¬ 
istics  of  the  dual  mode  system;  for  a  high  gain  servo  system 
subject  to  torque  saturation  can  be  designed  to  provide  auto¬ 
matically  the  characteristics  of  the  dual  mode  system.  To 
achieve  automatic  dual  mode  operation  the  switching  boundary 
used  for  the  relay  system  must  be  modified  somewhat  to  account 
for  the  effect  of  finite  gain  and  to  provide  a  linear  region. 

In  order  to  force  the  system  to  follow  the  optimum  trajectory 
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when  the  gain  Is  not  infinite,  it  is  necessaxy  that  torque 
saturation  occur  at  the  optimum  trajectory.  Hie  system  will 
then  follow  the  optimum  trajectory  until  the  linear  region 
is  reached.  Thus  zero  torque  must  occur  somewhat  ahead  of 
the  optimum  trajectory  to  insure  that  full  torque  is  obtedned 
at  the  instant  the  optimum  trajectory  is  reached.  This  can 
be  accon^jllshed  by  constructing  the  neutral  boundary  (locus 
of  points  in  the  e  vs  #  plane  for  •vrtiich  the  torque  is  zero) 
shown  in  Fig.  2.  This  neutral  boundary  is  constructed  by 
shifting  the  left  half  of  the  optimum  trajectory  to  the  left 
by  an  amount  ^  and  the  right  half  of  the  optimum  trajectory 
to  the  right  by  an  amount  A;  ■vrtiere  A  is  equal  to  the  value 
of  error  which  will  produce  torque  saturation^ 

Thus: 

A=  T/K  ;  (2) 


where 

T  =  Maximum  Torque  Output  , 

K  =  Gain  from  Summing  Point  to  Torque  Output 

The  two  halves  of  the  curve  are  then  Joined  by  the  straight 
line  whose  equation  is  e  +  te  -  0;  where  t  Is  chosen  so  that 
the  straight  line  is  tangent  to  the  two  paurabolas;  thereby 
providing  a  smooth  transition  between  the  nonlinear  aM  linear 
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portions  of  the  curve.  Thus  hy  using  this  neutral  curve,  ve 
have  obtained  our  goal  of  having  the  error  reduced  to  zero  In 
the  minimum  time,  while  still  maintaining  the  desirable  char¬ 
acteristics  of  a  linear  syston. 

The  equation  for  the  neutral  curve  Is  given  In  three 

parts: 


e  + 


e  <  -  ex  , 


e  +  re  =  0 


j  -ei  <  e  <  ei , 
;  e  >  ei  , 


(3) 


where  ei ,  the  value  of  e  at  vrtilch  the  linear  and  parabolic 
portions  of  the  curve  join,  and  t,  the  time  constant  In  the 
linear  region,  cqn  be  determined  as  follows.  At  the  point 
of  Intersection  to  the  left  of  the  origin: 


ei  +  ei‘ 


2T 

J 


where  ^  is  the  value  of  e  at  the  point  of  intersection. 


3 


2Tt 

J 


(5) 
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2Tt 

J 


Qua 

J 


(6) 


K^y- 


If  the  linear  portion  of  the  curve  is  to  he  tangent  to  the 
parabolic  protion  then  fei  miist  have,  only  one  value. 
Therefore^ 


( 


8m 

j 


(7) 


>&ere 


(8) 


and 


(9) 


and  ei  =  2A  .  (10) 

Ibe  equations  (8)  and  (.10)  give  the  values  of  t  and  ei  in  (3). 
This  neutral  curve  can  be  generate  by  producing  a  nonlinear 
function  of  e  such  that: 


f(e) 


e*  +  A 


•  /  2TA 
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f 


r 


,  zm  ^  ^  ,  zm 

i’*A  !  ■  (U) 

!Ibe  function  f(e)  Is  then  added  to  e  as  shown  In 
Fig.  3*  Since  the  torque  is  now  controlled  hy  e  +  f(e),  the 
torque  is  zero  when  e  +  f(e)  s  o  and  hence  the  correct  neutral 
curve  is  generated. 

The  function  f(e)  is  not  difficult  to  generate  since 
it  consists  of  a  linear  region  and  a  region  characterized  ty 
a  square  law  relation. 

While  it  may  seem  somewhat  arhitrary  to  have  chosen  t 
so  that  the  parabolic  and  linear  portions  are  tangent,  there 
are  some  definite  advantages.  Not  only  does  this  insure  a 
smooth  transition  between  the  linear  and  nonlinear  range,  but 
this  choice  guarantees  that  in  the  linear  range  the  system  will 
be  stable  with  a  danq>ing  factor  of  0.707*  This  can  be  easily 
shown  as  follows;  consider  the  system  shown  in  Fig.  3*  IQ  the 
linear  region. 


G(8) 


(1  *  TS)  K 
J8» 


(12) 
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^ere  G(8)  -•*  open  loop  tranefer  function. 

The  control  ratio  (closed  loop  transfer  function)  is 


C  .  G(s) 

T  “  TToTgr 


(1  *■  TS)  K 
Js* 


■[I  +  T8)  K 
Jsa 


(1  +  TS)  K 


Js*  +  (1+  ts)  K 


(13) 


The  poles  of  (s)  are  the  solutions  of  the  equation: 


or 


Js^  +  (1  +  ts)  K  -  0  , 

a  ^  ,  2K  ^  K 

^  *  V—  ®  ^  T  "  ° 


(14) 


This  can  be  written  as 


si*  +  2  S  8  +  0^*  *  0  ; 


where 


(») 
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aeserye  Torque 


The  foregone  discussion  assumes  that  the  system  Is 
of  second  order;  In  reality  there  will  always  he  other  time 
constants  present  In  any  practical  system.  When  there  are 
other  time  constants  present,  an  exact  solution  to  the  problem 
of  positioning  the  output  of  the  system  In  the  minimum  time  Is 
made  considerably  more  complicated.  In  this  case  It  would  be 
necessary  to  generate  neutral  curves  and  surfaces  In  n- 
dimenslonal  phase  space,  vhere  n  is  equal  to  the  order  of  the 
system.  However  if  the  gain  of  the  system  is  not  too  high 
the  effect  of  the  other  time  constants  can  be  accounted  for  by 
modifying  the  neutral  curve  In  the  phase  plane  so  as  to  pro¬ 
duce  control  which  is  approximately  optimum. 

Consider  the  case  of  the  neutral  curve  that  Is  not 
modified.  As  a  result  of  the  storage  effect  of  the  minor 
time  constants,  the  actual  trajectoiy  BCD  in  the  phase  plane 
would  overshoot  the  trajectroy  AO  ^irtiich  passes  through  the 
origin.  This  is  shown  in  a  somewhat  exaggerated  manner  in 
Pig.  4.  Since  the  full  torque  is  already  ejcpended,  there  is 
no  torque  available  to  return  the  system  to  AO;  with  the  re¬ 
sult  that  more  than  one  torque  reversal  is  necesseoy  in  order 
to  reach  the  equilibrium  point  in  the  phase  plane.  This  sit¬ 
uation  can  be  remedied  by  altering  the  neutral  curve  slightly 
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so  that  It  no  longer  has  the  same  shape  as  the  trajectory  in 
the  phase  plane.  If  the  neutral  curve  is  made  someidiat 
"flatter'*,  then  less  than  the  saturation  torque  is  sufficient 
to  move  -t^e  system  along  the  iq>per  saturation  limit.  The 
actvial  trajectory  vill  approach  the  neutral  curve,  with  the 
restilt  that  only  one  reversal  of  torque  is  requizad  to  reach 
the  equilibrium  point.  This  is  shown  (also  exaggerated)  in 
ng.  5. 

Notice  that  with  this  modification  the  ^plied  torque 
is  not  the  maxlmm  available  torque  over  the  entire  trajectory, 
but  is  reduced  to  a  value  slightly  below  maxtimm  for  part  of 
the  trajectory.  Full  torque  is  applied  only  ^en  the  actual 
trajectory  is  being  forced  toward  the  saturation  limits,  after 
which  slightly  less  than  full  torque  is  applied. 

Thus  we  have  provided  a  "reserve  torque"  which  is 
used  only  for  a  portion  of  the  trajectory.  Since  this  "reserve 
torque"  is  small,  full  torque  or  almost  full  torque  is  eiLways 
applied.  This  system,  is  only  silently  slower  than  an  optimum 
system,  in  which  full  torque  Is^  applied  for  the  entire  tra¬ 
jectory. 

This  reserve  torque  is  also  extremely  valuable  when 
there  are  random  dlstuzbances.  Suppose  a  disturbing  torque 
occurs  when  the  system  is  in  the  unsaturated  zone.  Without 


297  - 


any  reserve  torque  the  system  can  be  brou^^t  back  to  the  unsat¬ 
urated  zone  only  if  the  disturbing  torque  Is  in  the  same 
direction  as  the  control  torque.  With  reserve  torque  the  system 
is  brought  back  to  the  unsaturated  zone  irrespective  of  the 
direction  of  the  disturbing  torque. 

Saturable  Lag  Network 

In  most  practical  servo  systems  it  is  generally  desir¬ 
able  to  have  a  high  static  gain  so  as  to  keep  the  error  due  to 
loed.  disturbances  low.  However,  as  the  gain  is  made  Isorger  the 
minor  time  constants  beccxne  more  troublesome;  not  only  because 
they  cOTiplicate  the  problem  of  reaching  the  equilibrium  position 
in  the  minimum  time,  but  also  because  they  may  cause  the  system 
to  become  unstable  in  the  linear  region. 

A  system  could  be  made  to  have  high  static  gain,  and 
still  be  stable  in  the  linear  range,  by  adding  Integral  compen¬ 
sation  as  shown  in  Fig.  6.  However,  the  problem  of  obtsdning 
optimum  control  for  large  step  inputs  is  made  even  more  com¬ 
plicated.  Since  no  useful  puipose  is  served  by  allowing  the 
output  of  the  integrator  to  exceed  the  value  required  for  full 
output  torque,  the  difficulty  is  resolved  by  limiting  the  out¬ 
put  of  the  integrator. 

As  the  load  distvurbances  are  generally  much  smaller 
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than  the  maximum  output  torque,  the  Integrator  output  can  he 
limited  at  a  value  considerably  less  than  that  necessary  to 
provide  full  output  torque.  With  this  arrensemeht,  a  constant 
load  dlstvirbance,  which  Is  not  sufficiently  large  enough  to 
cause  the  Integrator  to  saturate,  will  not  produce  a  steady 
state  error.  However,  when  a  large  step  input  of  displacement 
Is  epplied,  the  ouiput  of  the  Integrator  will  be  much  smaller 
than  either  e  or  e,  and  hence  will  have  a  negligible  effect 
on  the  performance. 

An  Example 

In  order  to  demonstrate  the  feasibility  of  the  methods 
presented  In  this  paper,  they  were  applied  to  the  design  of  a 
control  system  to  stabilize  an  Inertial  platform.  A  system  of 
this  type  Is  generally  used  to  maintain  the  platform  at  a 
fixed  position  within  a  few  seconds  of  arc,  in  the  presence  of 
load  disturbances;  except  vhexi  the  platform  is  moved  to  a  new 
position,  at  which  time  the  input  is  a  stQ)  of  many  degrees  of 
eoc.  Hence  the  system  has  the  following  characteristics: 

1.  Extrmely  low  error  for  constant  load  torque  up  to 
30i?  of  fiill  output  torque. 

2.  Minor  time  cozustants  axe  presented. 

3.  The  system  is  subjected  to  step  inputs  of  displacement 
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hundreds  of  times  la3:ger  than  necesaaxy  to  produce 
torque  saturation. 

The  system  Is  shown  In  Fig.  7*  In  the  linear  region: 

8® (2  X  10"®  8+1)*  s®( 8+500)* 

(16) 

The  root  locus  plot  Is  shown  In  Fig.  8.  For  the  above  value  of 
K  the  closed  loop  poles  are  In  the  left  half  plane  and  provide 
a  reasonable  margin  of  stability.  However,  as  the  system  Is 
only  conditionally  stable,  InstablUty  will  arise  for  li^uts 
large  enough  to  cause  saturation.  If  only  linear  cGnq>ensatlon 
Is  used. 

The  Integrator  output  was  limited  at  a  value  which 
provides  slightly  more  than  30^  of  full  output  torque,  thus 
satisfying  the  requirement  of  low  error  for  load  torques  vp 
to  30^  of  full  output  torque.  The  function  f(e)  was  first 
generated  to  provide  automatic  dual  mode  operation  without 
provision  for  "reserve  torque". 

The  system  was  simulated  on  an  analog  conputer. 

Fig.  9  Illustrates  the  response  to  a  step  liput  of  displacement 
one  hundred  times  larger  than  necessary  to  produce  saturation. 
The  corresponding  phase  plane  plots  with  liputs  of  different 
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magnitudes  are  Illustrated  in  Fig.  10.  These  curves  shofvr  that 
two  reversals  of  torque  are  required  before  the  linear  region 
is  reached.  The  extra  torque  reversal  is  a  consequence  of  the 
two  minor  time  constants  located  at  T  =  2  x  10”®  sec. 

The  perfoimance  of  the  previous  system  was  inqiroved 
by  providing  "reserve  torque".  This  was  accomplished  by  mul¬ 
tiplying  f(fe)  by  a  factor  of  1.13  before  adding  it  to  e,  thus 
producing  the  necessary  "flattening"  of  the  neutral  curve. 

The  step  response  (Fig.  11)  and  phase  plane  plot  (Fig.  12)  show 
that  only  one  torque  reversal  is  now  required  in  order  to  reach 
the  lineeo:  region.  It  is  to  be  noted  that,  even  with  this  pro¬ 
vision  for  "rerverse  torque,"  full  torque  is  provided  for  almost 
the  entire  trajectory,  and  the  settling  time  in  Fig.  10  is  about 
shorter  than  that  in  Fig.  8. 

Step  respOhses  and  phase  plane  plots  were  also  ob¬ 
tained  for  a  second  system  employing  linear  conq)ensation  without 
limiting  the  integration  (Fig.  13a  and  llj-);  for  a  third  system 
eo^loying  linear  conqpensation  with  the  integration  limited  (Fig. 
13b  and  1^);  and  for  a  fourth  system  vising  the  non-linear  neutral 
curve,  but  with  the  lnte£p:sitlon  not  limited  (Fig.  13  c  and  I6). 
Conparlson  of  these  curves  with  those  obtained  for  the  previous 
system  clearly  illustrates  the  svqierlorlty  of  the  system  which 
eoqploys  tbe  non-Unear  curve  and  has  the  integration  limited. 
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These  curves  show  that  the  second  system  Is  unstable  for  laz^e 
Inputs,  and  that  the  third  and  fourth  systems  require  a  much 
longer  time,  and  several  more  torque  reversals  before  settling. 

The  Importance  of  the  Integration  In  the  system  Is 
Illustrated  by  the  response  to  a  st^  Input  of  torque  disturb¬ 
ance.  ”*‘rf  the  Integration  Is  removed  then  a  step  Input  of 
torque  disturbance  results  In  a  steady  stqte  error  (^g.  17m); 
vhlle  no  steady  state  error  results  \dien  the  Integration  Is 
present  (Fig.  17b). 

The  performance  of  a  dual  mode  system  Is  svgperlor  to 
a  relay  system  since  the  automatic  dual  mode  system  behaves 
as  a  linear  system  when  subjected  to  a  random  load  disturbance 
which  does  not  exceed  the  maximum  torque  output;  with  the  re¬ 
sult  that  the  error  Is  very  much  less  than  would  be  obtained 
If  the  system  were  of  the  relay  type.  IMs  can  be  seen  from 
Inspection  of  Fig.  18  which  shows  the  response  of  the  auto¬ 
matic  dual  mode  system  to  a  torque  disturbance  which  Is  an 
arbitrary  function  of  time. 

CONCLUSION 

This  paper  hac  presented  several  simple  methods  for 
obtaining  approximately  optimum,  control  In  a  practice  system 
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with  minor  constants.  These  give  approximately  the  response 
of  an  optimal  hang-bang  system  with  large  inputs^  while  still 
maintedning  the  desirable  response  of  a  linear  system  to  ran¬ 
dom  torque  disturbances.  The  required  control  function,  which 
consists  of  a  linear  region  and  a  region  characterized  by  a 
square  law  relation,  can  be  generated  by  the  use  of  a  few 
diodes. 

The  results  of  an,  Analog  Coc^}uter  study  demoz^strate 
the  value  of  the  methods  proposed  in  the  paper.  Conparisons 
of  the  perfoimance  of  a  system  incorporating  these  methods 
with  the  perfoimance  of  several  other  systems  illustrate  how 
the  proposed  methods  inprove  the  stability  and  reduce  the 
settling  time. 


NOTEt  The  research  contained  in  this  report  has  been  sponsored 
by  the  Air  Fbrce  Office  of  Scientific  Research,  Office  of  Aerospace 
Research  (USAF),  Washington  2^,  D.C.  ;  under  Grant  No.  AF-AFOSR-62- 
321  and  its  predecessor  Contract  No.  AF  49(638)-^86. 
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Figure  2.  Neutral  Cur^e  for  Dual  Mode  System  With  Finite  Gain 
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Figure  3.  Second  Order  Automatic  Dual  Mode  System 


Figure  4.  Effect  of  Minor  Time  Const^s  on  Phase  Plane  Trajectory 
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Figure  5.  Modified  Neutral  Curve 

(To  Obtain  "Reserve  Torque") 
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Figure  7.  System  Used  for  Analog  Computer  Study 
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Figure  10.  Phase  Plane  Plot  for  System  Without  Reserve  Torque 


Figure  11.  Step  Response  of  System  Incorporating  Reserve  Torque 
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Figure  12.  Phase  Plane  Plot  of  System  Incorporating  Reserve  Torque 


a)  Linear  Compens^lon  and  Integration  Not  Limited 

b)  Linear  Compensation  and  Integration  Limited 

c)  Non-Linear  Compensation  and  Integration  Not  Limited 


Figure  13.  ^ep  Response 
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Figure  16.  Phase  Plane  Plot  for  System  With  Non-Linear 
Compensation  and  Integration  Not  Limited 


torque  disturbance 


Figure  18.  Response  to  Random  Torque  Disturbance 
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ABSTRACT 


Much  attention  has  been  devoted  in  recent  years  to  the  problem  of  the 
optimum  control  of  a  system  vhere  the  manipulated  variable  is  subject  to  a 
saturation  limitation.  This  is  termed  optimum  nonlinear  control.  Much  of  the 
effort  in  this  area  belongs  to  the  minimal  time  school  of  Bellman,  Pontryagin 
and  others.  In  1944  the  author  began  a  theoretical  and  experimental  study  of 
optimum  nonlinear  control  vhere  it  is  desired  to  produce  response  that  is  best 
in  every  practical  engineering  sense.  This  work  is  discussed  here,  namely  the 
theory  as  far  as  appears  practical  for  engineering  applications,  limitations 
on  applicability,  and  examples  of  controllers  patented  and  produced  on  the 
basis  of  the  theory. 


INTRODUCTION 


When  a  hydraulic  governor  is  connected  to  the  throttle,  or  eouivalent 
member,  of  a  prime  mover  this  throttle  is  stroked  by  the  servomotor  of  the 
governor.  The  governor  output  is  the  piston  of  the  servomotor.  This  is  con¬ 
nected  to  the  throttle.  The  speed  m*  of  this  piston  depends  on  the  opening  of 
the  control  valve  metering  oil,  or  other  liquid,  to  the  servomotor  cylinder. 
Normally  this  valve  is  of  the  spool  type  where  the  position  of  the  valve 
plunger  determines  the  valve  opening.  Pull  opening  of  this  control  valve  cor¬ 
responds  to  the  maximum  speed  of  the  servomotor  piston,  in  fact  one  can,  for 
practical  purposes,  make  this  piston  travel  at  maximum  speed  in  either  direc¬ 
tion  by  moving  the  valve  plunger  throu^  full  stroke  to  one  end  or  the  other. 
Servomotors  are  usually  built  so  that  the  maximum  servomotor  piston  speed  is 
the  same  in  each  direction.  For  convenience  this  is  assumed  here.  The  problem 
of  optimum  nonlinear  control  is  understood  to  be  that  of  bringing  the  prime 
mover  to  equilibrium  speed  in  the  best  manner,  assuming  that  it  is  initially  in 
a  nonequilibrium  state.  Although  the  prime  mover  is  in  the  initial  nonequl- 
librlum  state  because  of  disturbances  on  the  system  of  which  it  is  a  part,  it 
is  assumed  that  the  sjrstem  is  undisturbed  after  it  leaves  the  initial  nonequi- 
librium  state  until  eouilibrium  is  attained.  Thus  if  the  load  is  chopped  on  an 
engine  under  isochronous  control,  although  the  engine  is  at  eouilibrium  speed, 
it  experiences  an  initial  acceleration  proportional  to  the  amount  of  load 
dropped.  See  Fig.  1.  It  is  now  desired  to  return  the  system  to  eouilibrium 
with  the  maximum  error  £  ,  the  duration  6  of  the  transient,  the  area  between 
the  error  curve  and  the  time  axis,  and  other  numerical  quantities  minimized 
simultaneously.  In  Fig.  1  the  quantity  c  is  the  error  in  the  controlled  vari-^ 
able,  i.e.  engine  rpm  (revolutions  per  minute)  error,  and  t  is  time.  In  1944 
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the  writer  analyzed  the  Woodward  governors  on  Hamilton  Standard  propellers. 

Here  the  aircraft  engine  speed  is  regulated  by  varying  the  propeller  pitch. 

This  pitch  corresponds  to  the  position  of  the  piston  of  the  hydraulic  servo¬ 
motor  built  into  the  hub  of  the  propeller  .  The  maximum  piston  speed,  and  hence 
rate  of  change  of  propeller  pitch,  is  limited  to  a  value  depending  on  pilot 
comfort  and  other  factors.  The  writer  solved  the  problem  of  optimum  nonlinear 
control  of  such  engine -propeller  units,  and  subsequently  carried  out  the  theory 
for  the  control  of  one  variable  as  far  as  he  felt  practical  considerations 
would  permit. 

The  following  questions  arise, 

1.  Is  there  always  a  unique  transient,  optimum  in  every  reasonable 
sense? 

2.  If  there  is  a  best  transient  can  it  be  obtained  by  operating  the 
rate  m*  only  at  its  maximum  numerical  value  or  zero? 

3.  If  there  is  an  optimum  transient  does  there  exist  a  function,  or 
set  of  functions,  of  the  controlled  variable  c  and  quantities  ob¬ 
tained  by  mathematical  operations  on  c  so  that  a»king  the  rate  m* 
depend  only  on  this  function,  or  these  functions,  the  optimum 
transient  will  be  obtained  autoMtically? 


THEORY 


Consider  the  eouations 


c'  ♦  <*.c  ■  Q 

(1) 

T  q'  +  q  ■  m 

(2) 

iiAiere*  cX  and  T  are  positive  numbers,  while  c’  and  q*  are  the  derivatives 

dc/dt  and  do/dt  of  c  and  q  respectively  with  respect  to  time  t.  The  quantity 
^need  not  be  a  constant  but  may  be  a  function  of  time  t.  Physically  q  may  be 
taken  to  be  engine  torque  or  proportional  to  it,  c’  as  rpm/sec,  c  as  rpm,  ^c 
as  damping  torque,  and  m  as  throttle-torque  time  constant.  The  author  Cl]  found 
early  in  his  studies  that  the  damping  term  oC  c  may  be  neglected  for  the  magni¬ 
tudes  of  -M.  that  occur  in  engineering  practise,  as  far  as  qptimum  nonlinear 
control  is  concerned.  This  results  in  a  great  reduction  ih  the  mathematical 
complexity  of  the  problem,  without  adversely  affecting  the  results. 

The  saturation  limit  on  the  derivative  m'  of  the  manipulated  variable  m  is 
equivalent  to 

l«'i  ^  K2  (3) 

for  a  positive  number  K2  and  the  absolute  value  [m'  l  of  m  '. 

/ 
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The  answer  to  the  three  ouestions  above  is  ”yes**  for  the  case  where  T  »  0* 
This  is  also  true  of  the  initial  conditions  likely  to  occur  in  practise  in  the 
case  where  T  /  0*  Some  initial  conditions  exist  in  this  last  case  where  opti¬ 
mum  transients  (minimum  overawing  and  underswing^  duration,  etc.)  can  be  at¬ 
tained  only  by  operating  ImM  below  saturation  during  at  least  part  of  the 
transient,  i.e. 


lm*l  -c  iU) 

It  is  particularly  important  that  the  answer  to  the  third  question  be 
"yes”.  For  the  automatic  control  of  the  variable  c  the  controller  usually 
must  operate  on  c  only  and  from  this  information  vary  m  to  bring  the  system  to 
eouilibrium.  It  is  thus  not  convenient  to  have  the  variable  m  depend  on  time  t 
explicitly. 

We  define  the  absouare  £x|  of  a  variable  x  by 

-  |xlx  (5) 

Thus  w  is  the  sighed  square  of  x.  This  function  comes  up  automatically  in 
optimum  nonlinear  control  where  it  plays  a  significant  role. 

In  as  yet  unpublished  work  the  writer  proved  that  optimum  nonlinear  con¬ 
trol  of  the  system  described  by  relations  (l),  (2)  and  (3)  with  ^  -  0  is  ob¬ 
tained  by  letting 

m*  *  -  K  X.  (6) 


where  K  is  arbitrarily  large  (♦  co)  and  X  is  the  control  function  given  by 


1  -  (1  > 


where 


and 


*  c  ♦  T  c  • 
sgn  y'  ■  1,  0,  -1 

according  as 

y*  >  0,  0,  or  0 

When  the  log  term  in  ^  is  imaginary  it  is  to  be  dropped, 

In  view  of  the  limit  (3) 


) 


1(7) 


(8) 

(9) 

(10) 
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when 


^  >  0 
s:  <0 

C  -  0 


(11) 


m* 


•  K. 


For  a  load  rejection  the  resulting  transient  is  as  shown  in  Fig.  2.  Similar 
curves  apply  for  other  initial  conditions.  The  quantities  t^,  t2  and  ty  de¬ 
note  switching  instants  at  which  in  practise  SI  changes  sign.  For  t  >t3  we 
have  C»  0  and  the  system  is  in  eouilibrium  with  ra*  •  0.  From  t  «  0  to  t^  we 
have  H  >0  whence  m*  *  -K2.  From  t  ■  t]^  to  t  «  t^  we  have  X-c.0  whence 
m*  ■  ^2*  Similarly  from  t  ■  t2  to  t  ■  t^  we  have  %  >0  and  m*  =  “^^2*  After 
t  *  t^  the  relation  ILwO  holds. 

If  the  lag  T  is  zero,  the  times  t2  and  to  coincide.  In  any  case  every 
other  solution  lies  above  the  curve  of  Fig*  Z,  and  is  thus  worse  in  every 
sense,  or  this  solution  crosses  the  curve  of  Fig.  2  after  the  point  where 
t  *  ti,  or  leaves  this  ciirve  plunging  below  it.  In  the  latter  case  the  curve 
reaches  eouilibrium  after  t  -  t^  so  that  the  duration  of  the  transient  is 
greater  and  this  curve  is  worse  than  that  of  Fig.  2  in  every  reasonable  engi¬ 
neering  sense, 

If  the  system  to  be  controlled  is  of  higher  order  than  that  of  eouations 
(1)  and  (2)  the  control  functions  involve  the  third  and  higher  order  deriva¬ 
tives  of  c.  Since  it  is  not  possible  to  obtain  such  derivatives  in  the  physi¬ 
cal  world  the  treatment  of  such  high  order  systems  is  largely  academic. 

If  a  system  has  two  or  more  dominant  lags,  as  ^^t)en  T  splits  up  into  two 
time  constants  and  T2  with  *  T2  the  improvement  that  can  be  obtained  by 
going  to  optimum  nonlinear  control  is  negligible,  and  in  the  opinion  of  the 
writer  not  economically  justifiable.  In  any  case  theory  and  experiment  indi¬ 
cate  that  it  is  advisable  to  lump  the  lags  of  a  sj'^stem  (the  lag  due  to  the 
damping  term  #c  being  neglected). 

To  obtain  stability  it  is  convenient  to  have  a  linear  band  of  operation 
near  eouilibrium.  It  is  advisable  to  use  the  control  function 

m'  =  -  K  [c  M  )(“  ♦  /Jlc'l)  c']  (12) 

for  positive  constants  /5  and  K.  The  value  of  is  chosen  to  give  good 
system  stability  in  the  absence  of  large  disturbances.  Then  fl  should  be  ad¬ 
justed  so  that  /^Ic'imax  dominates  y  for  the  maximum  value  Ic'imax 
to  be  encountered  in  practise.  Practical  experience  shows  that  It  is  desir¬ 
able  to  have 

^  25 y  (13) 

A  curve  as  in  Fig.  1  was  obtained  in  the  laboratory  for  a  case  where  the 
system  lags  have  time  constants  of  0,4  sec.,  0.1  sec,,  0,1  sec.,  and  0,05  sec. 
respectively.  A  very  great  variety  of  physical  systems  and  initial  conditions 
were  tested  in  the  laboratory  by  the  author  and  his  associates  oyer  a  period  of 
several  years,  while  hardware  was  developed  and  patents  obtained  to  exploit 
the  results. 
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A  reasonable  compromise  of  the  precise  theory  may  be  attained  by  letting 

kl,  •r' 


Z  •  + 


for  an  average  value 


2KiK 


(14) 


1^2 


for  the  responses  to  larger  disturbances,  so 


that  the  ngnlinear  control  function  3l  of  formula  (?)  is  replaced  by  the  linear 
function  X  of  relation  (!/*)• 


Tests  showed  that  in  practise  one  can  always  adjust  K  and  in  ^  of  for¬ 
mula  (12)  with  /3  »  0  so  that  the  raaxiimim  error  for  the  worst  /idden  distur¬ 
bance  encountered,  such  as  a  full  load  change  on  an  engine,  is  the  same  as 
when  optimum  nonlinear  control  is  used.  Also,  the  curves  are  practically  the 
same  in  their  entirety.  If  the  magnitude  of  the  disturbance,  such  as  load,  is 
reduced,  we  can  reach  a  point  where  the  error  is  half  as  large  when  a  nonlinear 
control  function  ^  is  employed.,  In  any  case  we  can  halve  the  area  between 
the  c  curve  and  the  t  axis.  Sec  Fig.  3  vhere  the  two  peaks  are  for  identical 
partial  load  changes  on  the  same  diesel  engine  but  the  control  is  linear  or 
nonlinear  as  noted.  Here  best  linear  and  nonlinear  control  are  compared  where 
for  larger  disturbances  these  controls  yield  practically  the  same  results.  If 
we  go  to  smaller  disturbances  the  responses  for  the  linear  and  nonlinear  ^ 
become  identical  for  practical  purposes. 


The  extravagant  claims  of  improvement  by  going  from  linear  to  nonlinear 
control  by  others  is  based  on  comparing  optimum  nonlinear  results  with  certain 
linear  controls,  but  not  the  best  that  can  be  obtained  by  the  use  of  linear 
control  functions.  How  then  can  the  great  amount  of  research  on  optimum  non¬ 
linear  control  be  Justified?  One  reason  is  that  with  the  optimum,  nonlinear 
control  theory  on  hand  the  engineer  can  tell  how  far  an  actual  design  deviates 
from  the  best,  and  he  can  modify  the  design  accordingly.  Also  the  disturbances 
that  occur  most  freouently  may  be  the  intermediate  ones,  v.»here  a  two  to  one 
improvement  with  nonlinear  control  is  attained,  A  customer  will  often  pay  for 
this  magnitude  of  improvement,  especially  in  military  and  snace  anplications. 
Finally,  the  nonlinear  control  equipment  is  sometimes  less  expensive  than  the 
correstx)nding  linear  device. 


TWO  GR  MORE  CONTROILLD  VARIABLES 

The  work  of  the  author  and  his  associates  [l]i  [2]  has  been  primarily 
restricted  to  the  control  of  one  variable.  Early  work  on  bang-bang  control  by 
D.  McDonald  [3},  A.  M.  Hopkin{4]  and  otiiers  was  a  Iso  restricted  to  one  con¬ 
trolled  variable.  The  approach  of  the  writer  is  being  extended  to  two  and. 
more  variables#  Recently  there  has  arisen  the  "minimal  time"  school  of  Bell?- 
man  [5l  ,  Kalitan  [6],  Pontryagin  [?]  and  others  involving  a  very  large  number 
of  workers  in  the  United  States,  the  Soviet  Union  and  elsewhere.  This  school 
is  concerned  with  systems  in  one  or  more  controlled  variables  with  manipulated 
variables  subject  to  saturation  limits,  where  it  is  desired  to  bring  the 
^stem  from  an  initial  state  to  another  state  in  such  a  way  as  to  minimize  the 
duration  of  the  transient,  or  minimize  (maximize)  some  other  functional  of 
time  and  the  controlled  and  manipulated  variables .  The  t  heory  does  not  apply 
to  minimizing  the  maximum  error.  This  is  the  performance  index  one  is  normal¬ 
ly  TOst  concerned  with  in  practise,  thus  the  user  wishes  to  know  whether  the 
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control  will  be  accurate  to  l/lQ$,  1%  or.  some  other  value.  He  usually  does 
not  care  about  minimizing  response  time.  Actually,  the  user  looks  at  the 
entire  response  curves  in  determining  the  goodness  of  a  control.  The  situa¬ 
tion  is  analogous  to  the  determination  of  beauty  queens.  It  is  not  enough  to 
make  a  finite  number  of  numerical  measurements  and  base  the  selection  on  these 
only,  such  as  on  the  dimensions  35-25-35*  A  response  curve  has  infinitely 
many  numerical  properties.  The  approach  of  the  writer  differs  from  the  mini¬ 
mal  time  school  in  that  he  looks  at  the  complete  system  responses  rather  than 
some  nuinerical  properties  of  these  responses,  selected  for  mathematical  or 
other  convenience.  On  the  other  hand,  the  great  value  of  the  minimal  time 
school  approach  is  that  by  minimizing  time  one  also  often  minimizes  every 
other  reasonable  measure  of  goodness  simultaneously.  On  the  other  hand  there 
are  sometimes  infinitely  many  distinct  means  for  bringing  the  given  system 
from  the  initial  state  to  the  final  state  in  minimal  time,  and  additional 
theory  must  be  employed  to  select  from  these  and  perhaps  other  solutions  one 
that  is  best  in  every  reasonable  engineering  sense. 

Consider  the  system  described  by  the  equations 


^1  '  ®1  ^  ®2 


^2  ‘  ®1  •  ®2 


(15) 


f  I 

for  controlled  variables  and  C2  with  time  derivatives  and  C2  respective¬ 
ly,  and  manipulated  variables  and  m2  where  the  magnitudes  |m{l,  lm2  |  of  the 
derivatives  mj,  m2  satisfy 


Imil  ^  1 

K  I  ^  ^ 


(16) 


Let 


Cl  ♦  C2 


-  Cl  -  C2 


(17) 


idle  nee  equations  (15 )  are  equivalent  to 

■  2  mj^ 

^2  *  ^ 


(18) 


We  wish  to  bring  the  system  from  an  initial  state 

(ci,  C2,  Cl,  C2)  “  (cjo*  ®20*  ®1,0*  ®2o) 


(19) 
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(20) 


for  real  numbers  c{q,  cgo  to  the  eouilibrium  state 

(ci»  C2,  c^,  C2)  -  (0,  0,  0,  0) 


in  best  manner  with  and  mo  subject  to  the  saturation  condition  (I6), 
o  asaimed  that  the  initial  state  is  not  eouilibrium.  If  the  variables 

and  C2  are  brought  to  eouilibrium  (ci  *  0,  cp  «  O)  in  a  minimal  time  M  it 
follows  that  C'l  and  C2  are  brought  to  the  eouilibrium  (C]_iK  0,  C2  *  0)  in  this 
time  M.  To  simplify  the  argument  aappose  that  cjjj-  C20*  0,  whence  C20*  0. 


The  first  eouat ion  (18)  with  the  first  condition  (I6)  describes  a  single 
controlled  variable  system.  For  this  the  optimum  transient  is  as  shown  in 
Fig.  4a  where  the  duration  of  the  transient  is  given  by 


Ml 


(1  V2)  Cjo 


‘=10  *  <=20 


(21) 


For  the  variable  Cg  there  is  a  second  optimum  transient  shown  in  Fig.  6b  where 
the  transient  duration  M2  is  given  by 


(1  +  \IT)  d 


■20 


^20  “  <=10  “  <=20  (^2) 


Suppose  that 


whence 


I  '  - 

c-|  C5  >  0 


C.n  >  ^on  >  0 


-10  ^  20 

The  curves  in  Figs.  4a  and  b  are  plotted  to  the  same  time  scale.  Clearly 


(23) 

(24) 


>  M2 


(25) 


whence 

“  “  %  (26) 

In  view  of  the  ineouality  '(25)  there  are  infinitely  solutions  of  C 

versus  time  with  fmijl  not  at  its  maximum  value,  which  bring  Cp  to  eoui¬ 
librium  in  the  time  Mj^.  It  follows  that  there  are  infinitely  solutions 
ttrat  bring  cj_  and  C2  to  equilibrium  in  the  minimal  time  M.  Thus  the  o-itimum 
nonlinear  control  problem  is  not  solved  by  minimizing  the  duration  of  the 
transient.  We  are  currently  investigating  this  problem. 

The  woric  of  the  minimal  time  school  must  be  assessed  on  the  basis  of 


\ 


33d 


engineering  as  well  as  mathematical  considerations.  It  is  one  of  many  possi¬ 
ble  approaches  to  optimum  control,  particularly  attractive  because  of  its 
mathematical  elegance. 

Recently  N.  P.  Smith  arid  the  writer  solved  the  problem  of  the  optimum 
nonlinear  control  of  a  system  where  a  single  control  l\inctiori  optimizes  the 
response  to  all  or  almost  all  disturbances  that  may  occur.  Thus  this  func¬ 
tion  applies  to  ramps,  sinusoids,  etc.  as  well  as  step  changes. 


APPLICATICWS 


A  practical  scheme  for  achieving  optimum  nonlinear  control  is  Shown  in 
Fig.  5.  This  involves  a  nonlinear  device  composed  of  a  ^arp  edged  orifice  in 
series  with  a  linear  (cylindrical)  pipe  section.  Vfe  let  pj^,  P2  and  p^j  be  the 
pressures  of  the  liquid  ahead  of  the  linear  section,  before  the  sharp  edged 
orifice,  and  after  it  respectively.  For  a  constant  a 

Pj^  -  P2  ■  a  Q  (27) 

where  Q  is  the  volumetric  rate  of  flow  through  the  unit.  The  pressure  drop 
P2  -  p^  across  tbe  sharp  edged  orifice  satisfies 

P2  -  P3  “  b  (q)  (28) 

for  the  absquare  and  a  constant  b.  It  follows  that 

Pj^  -  P3  -  a  Q  ♦  b  (qV  (29) 

When  Q  is  small  b^Q)  becomes  K-jQ  for  a  constant  whence  relation  (29)  is  the 
linear  expression 

Pj^  -  p^  -  (a  ♦  Kj)  Q  (30) 

This  principle  is  used  in  Fig.  5  in  the  case  of  a  hydraulic  speed  governor  for 
prime  movers.  The  motion  of  the  bellows  output  (B-bellows)  is  assumed  to  be 
small.  The  output  m  is  pB*oportional  to  the  pressure  p^^.  Let  k-^  and  y  be  the 
araa  and  coordinate  of  the  free  end  of  the  A-bellows.  we  land  area  kj  is 
small  so  that 

*1  y'  “  Q  "  (31) 

for  practical  purposes.  Let  be  the  scale  of  the  A-bellows  spring.  It 
follows  that 

Aj^  P3  -  K3  y  (32) 


The  pilot  valve  is  constructed  so  that 
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for  the  coordinate  x  of  the  valve  plunger  and  flow  coefficient  K,  .  By  rela¬ 
tions  (31)  and  (32) 

Ai^ 

Q  “  <34) 

K3 


whence  by  relation  (29) 


•  *1^  .  1 .. 

Pi  “  P3  "  K,  P3  tP3l 

(35) 

The  force  eouation  for  the  valve  plunger  is  now 

Kj  X  -  c  ♦  A3  pj 

(36) 

for  constants  Kc  and  K/  and  rpm  c  of  the  ballhead.  By  eouations 
and  (36) 

(33),  (34) 

h  *^1  .  *^6  « 

A  K  K  ^  A3 

3  4  3 

(37) 

The  tine  constant  (KrAj^^/A-K.K^)  is  taken  small  so  that  eouation 

(37)  reduces 

n  -  ^6 

*^3  A 

*3 

(38) 

The  stroke  of  the  B-bellows  is  proportional  to  the  stroke  z  of  a  piston  i(^iich 
controls  the  rate  of  flow  of  liquid  to  the  servomotor  with  piston  coordinate 
m.  For  a  constant 

■'  ■  Ky  z 

(39) 

Also  for  a  constant  Kg 

*  “  Kg  Pi 

(40) 

By  relations  (35),  (38)  -  (40)  it  follows  that  eouation  (12)  is  satisfied 
where 

(41) 


/3 


s'*  h 


Th*  results  of  running  this  governor  [s]  on  a  General  Motors  (M-671  engine  are 
8ho%ffi  In  Fig.  3< 

An  electrical  version  is  pictured  in  Fig.  6.  The  triangles  represent 
operational  amplifiers.  The  absquare  is  obtained  by  the  use  of  a  thyrite  re^ 
sister  which  has  the  ^operty 


i  -  C  {v}  (42) 

for  the  current  i  through  the  thyrite,  voltage  drop  V,  and  a  constant  C.  To 
satisfy  relation  (42)  the  voltage  range  is  one  for  whidi  the  maximuin  value  is 
about  10  tines  the  mininiuni.  For  small  V  relation  (42)  becomes  linear.  In  the 
controller  of  Fig,  6  the  variable  c  is  differentiated  to  obtain  c*.  A  voltage 
y  proportional  to  c*  is  applied  to  the  thyrite  to  yield  a  curront  proportional 
to  {c*).  Signals  proportional  to  c,  c’  and  ^c*}  are  added  to  obtain  a  voltage 
proportional  to  the  control  function  ^  .  This  voltage  is  employed  to  stroke 
a  valve  in  the  usual  manner  [9]. 

A  magnitude  amplifier  waterwheel  governor  version  of  Fig.  6  was  developed 
for  production  [lo]. 

The  nonlinear  governors  discussed  here  are  illustrative  of  the  few  opti¬ 
mum  nonlinear  controllers  built  by  industry.  Although  there  are  areas  where 
the  Improvement  of  performance  Justifies  the  use  of  such  controllers,  for  most 
control  problems  linear  control  functions  are  still  indicated,  whether  or  not 
saturation  is  involved. 


OONaUSIOKS 


For  physical  systems  in  one  controlled  variable  with  one  dominant  lag  and 
the  input  subject  to  saturation,  a  two  to  one  improvement  in  the  maximum  error 
for  intermediate  disturbances  may  be  obtained  by  going  from  the  best  linear 
control  function  to  a  nonlinear  control  Ainction  involving  the  absquare ,  Sharp 
edged  orifices  and  nonlinear  electrical  resistors  make  the  practical  produc¬ 
tion  of  such  control  functions  possible.  Due  to  lags  and  other  factors  al¬ 
ways  present  in  physical  systems  more  improvement  cannot  be  obtained.  The 
writer’s  nonlinear  approach  has  the  advantage  that  all  reasonable  criteria  for 
optimum  response  are  satisfied  simultaneously.  This  includes  minimizing  the 
maximum  error  for  a  transient,  which  the  user  considers  a  most  important  prac¬ 
tical  criterion  of  goodness.  He  is  usually  not  concerned  with  minimizing  time. 
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Unfortunately  minimizing  the  maximum  error  is  excluded  from  the  class  of  f\xnc^ 
tions  and  functionals  treated  by  the  well-known  optimization  school  of  Pontry- 
agin,  Bellman  and  others.  For  the  case  of  two  or  more  controlled  variables 
the  minimal  time  approach  fails  in  selecting  the  optimum  nonlinear  response. 

N.  P.  Smith  and  the  writer  recently  found  a  single  control  function  to  give 
optimum  nonlinear  response  to  a  large  class  of  disturbances,  if  not  all.  Thus 
optimum  response  is  obtained  regardless  of  whether  or  not  the  disturbance  is  a 
step,  ramp,  sinusoid,  or  something  else. 
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FIGURE  1.  OPTIMUM  RESPONSE 


FIGUM  2,  OPTIMUM  RESPONSE  FOR  SYSTEM  WITH  TWO  LAGS 


THROTTLE 


FIGURE  5.  NONLINEAR  HYDRAULIC  GOVERNOR 


FIGURE  6.  NONLINEAR  ELECTRONIC  GOVERNOR 
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Abstract 


This  paper  is  a  brief  review  of  the  current  status  of  optimal  programming 
theory  as  applied  to  multivariable  control  systems  in  the  presence  of  noise. 
Open- loop  programming  of  control  variables  for  a  noise-free  non-linear  system 
is  discussed  first,  followed  by  a  description  of  closed-loop  gain  programming 
for  a  linearized  version  of  the  same  type  of  system.  Optimal  filtering  of 
noisy  measurements  is  then  reviewed  for  linear  time-varying  systems.  A 
combination  of  these  techniques  promises  to  provide  optimal  control  for  a  wide 
class  of  systems  and  performance  indices. 


OPTIMAL  PROGRAMMING  FOR  NONLINEAR  SYSTEMS  (NOISE-FREE) 

The  solution  of  optimal  prq|;raming  problems  on  computers  may  now  be 
considered  practical  and  straightforward  through  the  use  of  gradient  (steepest- 
ascent)  techniques  developed  within  the  last  few  years  (Refs.  1-7).  The  type 
of  problem  that  can  be  solved  is  as  follows: 


Given:  A  physical  system  governed  by: 


X  -  f(x,  u,  t) 


where 


f  (x. 


x(t) 

•  n 

by 

1 

matrix 

of 

state 

variables 

u(t) 

•  q 

by 

1 

matrix 

of 

control  variables 

x(t^) 

Given 

1,  t) 

«  n 

by 

1 

matrix 

of 

known 

functions  (non-linear) 

Find;:  u(t)  to  maximize 

0  [x(tj^)  r  »  ^1  ^ 


with  terminal  constraints 

♦  [x(tp  ,  t^]  -  0 

where 

9  (x,  t)  is  pay-off  function  (given) 

\|f  (x,  t)  is  an  r  by  1  matrix  of  given  functions,  r  ^  n  -  1  . 
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Solution:  Calculus  of  variations  using  steepest-ascent  numerical  calculations. 


This  is  an  open-loop  control  system  with  programmed  control  variables, 
u(t)  .  This  is  not  a  feasible  type  of  control  to  use  in  many  systems  due  to 
errors  in  the  determination  of  initial  conditions  and  parameters  of  the  system. 
However,  if  this  "path"  is  used  as  a  nominal  path  and  small  perturbations  are 
considered  around  it,  a  linear  feedback  control  system  with  programmed  gains 
can  be  synthesized  that  will  bring  the  system  very  close  to  the  desired  terminal 
conditions . 


NEIGHBORING-OPTIMUM  FEEDBACK  CONTROL  (NOISE-FREE) 

A  simple  approach  to  this  problem  is  what  might  be  called  "minimum 
incremental  effort"  control.  It  is  a  straightforward  application  of  a 
quadratic  performance  index  to  the  system  linearized  around  the  optimum  path 
of  the  previous  section.  This  is  discussed,  with  several  examples,  in  Refs. 

8  and  9.  A  slightly  more  sophisticated  approach  is  used  in  Refs.  10  and  11 
where  the  performance  index  is  the  same  one  used  in  the  original  nonlinear 
problem;  here  a  true  neighboring-optimum  is  found  using  the  second  variation. 
An  outline  of  the  type  of  problem  solved  by  this  latter  technique  is  as  follows: 

Given:  u(t)  that  minimizes  (J[x(tj^)  ,  t^]  with  i[x(t^)  ,  t^^^]  ■  0 

and  given  x(t^)  ,  for  x  *  f(x,  a,  t) 

Find :  6u(t)  such  that  u(t)  +  Su(t)  minimizes  ^[x(tj^)  ,  with 

♦  ,  tj^}  =  0  and  given  x(t^)  +  8x(t^)  for  x  =  f(x,  u,  t) 

Solution:  8u(t)  •  -  A(t,  t^)  Sx(t^)  ,  where  A  (t,  t^)  Is  pre-calculated 

from  second  variation. 


Note  I; 

Note  2: 


The  concept  Is  readily  extended  to  continuous  measurement  of 
8x(t)  ,  ^  8u(t)  ■  -  y\.(t)  Sx(t)  . 

8x(t}  due  to  Su(c)  governed  by  linear  equations: 


^  (8x)  -  F(t)  8x  +  G(t)  8u 


OPTIMAL  ESTIMATION  FOR  NOISY  TIME-VARYING  LINEAR  SYSTEMS 

In  Refs.  12-14,  a  practical  filter  was  developed  for  time-varying  linear 
systems  with  Gaussian  random  noise  in  the  system  and  In  the  measurements.  The 
procediire  may  be  regarded  as  keeping  track  of  the  mean  values  and  covariances 
of  a  multivariable  Gaussian  distribution  In  a  Harkov  process.  An  easy  way  to 
understand  the  optimal  estimation  procedure  Is  to  consider  only  one  step  of  a 
discrete  process  where  either  (I)  a  set  of  measurements  is  made  or  (2),  a  set 
of  discrete  forcing  functions  Is  applied.  First,  consider  the  improvement  in 
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the  estimate  as  the  result  of  a  set  of  measurements: 

Given:  (1)  A  priori  estimate  of  state  variables  and  covariances: 

elx]  •  X  (an  n  vector) 

c[(x  -  x)  (x  -  x)  1  ■  P  (an  n  x  n  symmetric  matrix) 
(2)  A  set  of  measurements  and  covariances: 
eCz]  ■  z  (a  p  vector) 

R  (a  p  X  p  symmetric  matrix) 


Find; 


Answer ; 


t[(z  -  z)  (2  - 


where 

z 


Hx 


Maximum  likelihood  estimate  x  ^  that  minimizes 


^  1  ‘T-l  .  1  p  *1  ^ 

0«jzR  z+-xP  x 


where 


O  =  (“)-(  )  . 

X  -  X  +  PH^  r"^  (I  -  H  x) 

P  -  P  -  PH^  (R  +  HPH^)"^  HP 

--1  T^-1  -1 

-  (P  +  H^R  H) 


where 

P  *  r[(x  -  x)(x  -  x)^] 

Next^  consider  the  degradation  in  the  estimate  as  the  result  of  a 
set  of  forcing  functions: 

Given ;  (1)  A  priori  estimate  of  state  variables  and  covariances 

c[x}  -  X 

e[ (x  -  X) (x  -  x)^]  »  P 


(2)  A  sec  of  forcing  functions  and  coyarlances: 
c{u]  -  u 

c[ (u  “  u) (u  -  u)^]-  Q 
where 

Ax  «  Gu 

Find ;  Revised  estimate  of  state  variables  and  covariances  after 
application  of  forcing  function. 
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Answer : 


X  *=  X  +  Gu 
p  =  p  +  Gqb^ 
where 

P  =  e[(x  -  x)(x  -  x)^] 

If  we  now  apply  the  above  to  a,  discrete  process  with  a  transition 

matrix  f  ,  that  describes  the  transition  from  the  state  at  the  (n-1)®^ 
n,n-l  '  ' 

step  to  the  n^^  step,  we  have  the  following: 

STOCHASTIC  CONTROLLER  FOR  DISCRETE  MEASUREMEOTS  AND  DISCRETE  FORCING  FUNCTIONS 
tn  ordinary  point 

t  1  X  -  +  K  (z  -  H  f  ,  X  1 )  if  measurement  made 
n,n-l  n-1  n  n  n  n,n-l  n*l 

f  ,  X  -  +  G  u  if  forcing  function  applied 
*n,n-l  n-1  n  n  ® 


T 

f  -  P  -  f  .  if  ordinary  point 
*n,n-l  n-1  n,n-l  ^ 

C(f  ,  P  1  f  ^  ^  R  ^  H  ]  ^  if  measurement  made 

^n,n-l  n-1  n, n-1  n  n  n** 

X  ^  j 

f  -  P  .  f  -  +  G  Q  G  if  forcing  function  applied 
‘n,n-l  n-1  *^n,n-l  n^  n  n  ^ 


T^-l 

K  -  P  H  R  ,  gain  in  optimal  estimator 

n  n  n  n  ^ 

In  the  limit,  as  the  process  and  the  measurements  become  continuous,  the 
results  of  Ref*  l4  are  obtained.  Instead  of  a  transition  matrix  we  then  have 
differential  equations: 

X  «  F(t)  X  +  G(t)  u 

In  place  of  covariance  matrices  we  have  the  correlation  matrices  R(t)  and 
Q(t)  that  serve  to  describe  the  **whlte  noise'*  in  the  measurements  and  in  the 
forcing  functions  respectively: 

cCzCt)  !  ■  z(t)  , 

e  ^[z(t)  -  i(t)3  [z(t)  ^  z(t)  ]^  5(t)  8(t  -  t)  , 

where  z  ■  Hx  ,  and 
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eru(t)l  “  u(t)  , 

C^CuCt)  -  u(t)}  [u(t)  -  u(t)]^|=  Q(t)  &(t  -  t) 

The  optimal  filter  then  becomes: 

^  ^  T  — 

x«Fx+Gu+PH  R  (z-  Hx)  ;  x(t  )  *  x 

'  o  o 

P  =  FP  +  PF^  -  PH^F^  HP  +  GQG^  ;  P(t^)  -  P^ 

where  _ 

€Cx(t^)]  = 

C  {[x(t^)  -  x^]  [x(t^)  -  1=  P^  . 

Ref.  15  discusses  the  estimation  problem  from  the  point  of  view  of  maximum 
likelihood  estimation  whlch^  in  turn,  is  a  type  of  "least  square"  fitting.. 

COWBINATION  OF  OPTIMAL  FILTER  AMD  GLCSED-LOOP  OPTIMAL  GAIM  FROGRAI^'-IMG 

In  Ref. 16  it  is  shown  that,  for  quadratic  performance  indices,  the 
combination  of  the  optimal  filter  and  the  closed- loop  optimal  gain  programming 
technique  (noise-free)  produce  optimal  control  for  linear  systems  in  the  presence 
of  noise.  There  is  reason  to  believe  this  combination  will  be  optimal  for  a 
more  general  class  of  performance  indices.  Using  the  nomenclature  of  the 
previous  sections  the  control  system  is  given  by 

u(t)  =  -A.it)  x(t) 

x  =  Fx  +  G(u+w) 

*  T  — .  1  — 

X  =  F  X  +  Gu  +  PH,  R  (z  -  K  x) 

P  =  FP  +  PF^  -  PH^  r‘^  HP  +  G^^ 

where 

e(w)  =  e(v)  =  0 

_  eCw(t)  w'^(T)]  =  Q(t)  S(t  -  T) 

€[v(t)  v^(t)]:  -  R(t)  8(t  -  t) 

where  ^(t)  is  a  set  of  pre-calculated  time-varying  gains  determined  from 
noise-free  control  considerations  regarding  x(t)  as  though  it  was  the  exact 
value  of  x(t)  .  Mote  P(t)  may  be  pre-calculated  and  the  time -varying  gains 
T  ~-l 

I((t)  ®  PH  R  stored  in  a  memory  for  use  during  operation  of  the  control 
system  (see  Fig. 1).  Obviously  this  system  can  also  be  combined  with  the 
neighboring -optimum  feedback  scheme  described  above. 


;  x(t  )  =  X  z  =  Hx  +  V 

o  o 

;  P(t  )  =  P 

^  O'  o 
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ABSTRACT 

A  method  is  presented  for  making  maximiim  likelihood  estimates  of 
the  state  variables  of  linear  and  nonlinear  dynamic  systems  over  a  finite 
time  interval,  utilizing  measurements  made  during  this  time  interval  and, 
if  available,  a  priori  estimates  of  the  initial  conditions  and  the  forcing 
functions.  For  a  linear  system,  the  filtering  results  are  identical  to  those 
of  Kalman  and  Bucy;  the  smoothing  result  is  an  extension  of  their  work. 
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statement  of  the  smoothing  problem 

Given:  (1)  A  dynamic  system  governed  by  nonlinear  differential  equations, 
X  »  f(x,  u,  t) 

where 

x(t)  s  n  X  1  matrix  of  state  variables, 
u(t)  s  mx  l  matrix  of  forcing  functions, 
f  a  n  x  1  matrix  of  known  functions  of  x,  u,  t 
t  =  independent  variable  (often  time) 


(2)  An  a  priori  estimate  of  initial  conditions, 

Elx(tj)]  ■ 

an  a  priori  estimate  of  errors  iii  x^, 

E  {lx(t,)  -  XqI  tx(t^)  -  *  Pq  ,  an  nxn  matrix 

(symmetric). 
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(3)  An  a  priori  estimate  of  the  forcing  functionsi 

E[u(t)]  s  u(t),  and 
an  a  priori  estimate  of  errors  in  u(t), 

E  |(u(t)  -  u(t)]  [u(t)  -  u(t)]'^|  X  Q(t)  6(t-T) 

(4)  A  set  of  measurements  for  t  <  t  <  t, . 

£  [  z(t)]  m  z,^t)r  a  p  X  1  matrix  of  functions, 

where  the  theoretical  relations  between  "exact*  measurements, 
z(t),  and  state  variables  x(t),  is  h[z(t),  x(t),  t]  =  0,  a  pxl 
matrix  of  known  functions,  and  an  a  priori  estimate  of  errors 
in  z(t), 

E  |[z(t)  -  ■z(t)]  [z(t)  -  ■i{T)]^|  X  R(t)  6{t-T) 

In  the  above  equations  E[  ]  means  "expected  value  of 
[  ]  ",  i.  e.  an  ensemble  average  over  many  identical  systems 
performing  simultaneously.  The  sumbol  6(t-T)  is  the  Dirac 
delta  function  and,  appearing  in  the  correlation  equations 
above,  it  indicates  "white  noise"  in  the  forcing  functions  and 
the  measurements.  In  effect  what  this  means  is  that  we  as¬ 
sume  the  correlation  times  of  the  noise  are  short  compared 
to  times  of  interest  in  the  problem.  The  sumbol  (  )T  indicates 
the  transpose  of  the  matrix  (  ). 

Find;  The  maximum  likelihood  estimate  of  u(t)  s  u(t)  and  the  initial 
conditions  x(t^)  ■  x^.  Note  this  gives  the  maximum  likelihood 

estimate  of  x(t)  a  x(t)  from 

X  =  f(x,  u,  t) 

x(t  )  =  X 
'o'  o 

SOLUTION  TO  THE  SMOOTHING  PROBLEM 
The  maximum  likelihood  estimate  will  be  obtained  by  minimizing 

(1)  j  =  7^0  Pq  ^  ^  Z  +tF  Q"^  u)  dt, 

t 

o 

where 

^  A 

()=()-  n 


with  the  constraints; 
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(2) 

(3) 


(4) 


(5) 


h(z,  X|  t)  a  0 
X  -  f(x,  U,  t)  a  0. 

Hit 

This  is  a  standard  problem  in  the  calculus  of  variations.  It  may  be 
solved  by  adjoining  the  constraints  (2)  and  (3)  to  the  expression  for 

J  in  (1): 


-  l~Tr5-l~  ^  1  -T--1  «  ^  1 -Ttt-I-.  ^  -T.A  ,,a  a 

X^  +  \  ^  ^  ^  U+X[x-  f  (x,  U, 


t)] 


+  /X  h(z 


.  X.  t)j> 


dt 


A 

Now  consider  variations  of  u  and  from  their  optimum  values: 

6  J  =  P  6x  +  r  <[z^r”^6z  +  u^  Q  ^  6u  +  X^[6^  -  6x  - 

o  o  o  J  I  ox  8u 


(6) 


+  m'^I 


8h  jjA  8h  jfAiT  - 

-s-  6z  +  — -  6xJ  /  dt 

8z  8x  J 


or. 


6J  =  x^  ^^o  '5x]^^  +  y  |(2^  R"^ 


J.  T  8h. 

+  (i  •5^)6z 


.  i  ;T  -T  8f  .  T  8h.  -a  ^-1  .t  8f, 


Let  us  now  choose  X(t)  and  p((t)  so  that 


(7) 


xT  +  xT 


8f 

■55E 


T  8h 

^  85E 


0;  X(t^)  =  0,  xj  -  x'^(t^)  X  0, 


(8) 


~T  — -1 
z^  R  ^ 


»■ 


Then  £qn.  (6)  become s» 

(9)  6J  X  y  -  x"^  |i)  6u  dt. 

t 

_ o 

♦See  for  example  Bryson,  A.  E.  i  and  Denham,  W.  F. ,  ”A  Steepest  Ascent 
Method  for  Solving  Optimum  Programming  Problems,"  Jour.  Appl,  Mech., 
June  1962,  pp.  247-257. 
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If  u(t)  is  the  optimal  estimate  of  the  forcing  functions  then  6J  *  0 
for  arbitrary  6u.  For  this  to  be  true,  it  is  necessary  that 


(10) 


Summarizing,  the  optimal  estimate  of  u(t)  and  x(t)  will  be  obtained 
by  solving  the  two  point  boundary  value  problem. 


(11) 

Diff. 

/  0 

A 

X  s 

Eqns. 

(12) 

X  a 

(13) 

Boun. 

(  *<'o» 

(14) 

< 

Cond. 

^(tf) 

Note  1:  If  no  a 

f(x,  u,  t)  with  u  =  u  +  Q(-|^)^  X  , 


81 


8h,T,  8h,-T--l  ^  - 


^  ^  ‘  with 


0. 


Note  1:  If  no  a  priori  estimates  o£  x(t  )  and  u(t)  are  given,  this  e 
senUally  means  P  Q  «o.  The  conditions  involving 

u,  P  and  5  above  are  simply  replaced  by,  ^ 


(15) 

(16) 


f  =  0 

du 


Mt,)  >  0 


Note  2:  A  simple  extension  of  this  analysis  would  be  to  make  R  and  Q 
functions  of  x  and  u. 


Note  3:  Another  simple  extension  would  be  to  include  estimation  of 
some  parameters  of  the  system. 


SMOOTHING  FOR  LINEAR  DYNAMIC  SYSTEMS 
if  the  system  is  linear,  it  can  be  put  into  the  form, 

(17)  X  a  F(t)x  +  G(t)u 

(18)  z  =  H(t)x  +  w(t) 

where  w(t)  is  a  known  set  of  bias  functions.  The  smoothing  problem 
then  simplifies  to  the  following: 
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iix 


(19) 

d 

A 

X 

F  ,  GQG^ 

A 

X 

Gu 

(20) 

dt 

X 

h^r'^h, 

X 

+ 

-H^R"^(z-  w) 

with  boundary  conditions, 

(21)  x(t  )  -  P  X(t  )  a  X 

'  o'  o  '  o'  o 

(22)  X{tj)  =  0 

This  two-point  boundary  value  problem  can  be  solved  in  a  straight¬ 
forward  manner  due  to  the  linearity  of  eqns.  (19)  arid  (20)  as  follows: 

Let  Xp(t)  and  ^^(t)  be  solutions  of  Eqns.  (19)  and  (20)  with  initial 
conditions 

(23)  .  ^(,^). 

(24)  Xp(t^)  =  0. 

Let  ^  (t)  and  *  (t)  be  nxn  matrices  which  are  solutions  to  the 

X  A 

homogeneous  version  of  Eqns.  (19)  and  (20),  i.e.,  with  u  a  z-  w  =  0, 
with  initial  conditions, 


(25) 

W 

=  Po' 

(26) 

W 

=  unit  matrix  a 

It  follows  that 

(27) 

II 

Xp(t)  +  ^(t)  Mt,) 

(28) 

X(t)  a 

Xp(t)  +  X(t^) 

In  order  to  satisfy  the  boundary  condition  (22),  we  must  have 

(29)  X(t^)  =  -  t45^(Vr^Xp(9 
Combining  (29)  with  (27),  (28)  and  the  relation, 

(30)  u(t)  a  u(t)  +  QG^  X(t)' 
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we  have  finally^ 


(31)  x(t)  .  Xp(t)  -  yt)  Xp{tj) 

(32)  u(t)  »  u(t)  +  QG^^\p(t)  -  4^(t)  [  VVli 

(33)  z(t)  »  Hx  +  w 


This  is  the  solution  to  the  smoothing  problem,  i.  e. ,  the  problem  of 

finding  the  maximum  likeliKopd  estimates  of  x{t),  u(t),  and  z(t)  over 

an  entire  interval  t  <  t  <  t.. 

o  —  —  f 

The  filtering  problem  is  concerned  with  finding  the  maximum  likeli¬ 
hood  estimate  of  x(t£)  only,  and  the  prediction  problem  is  concerned 
with  finding  the  maximum  likelihood  estimate  of  x(t)  for  t  >  t^.  Ele¬ 
gant  solutions  to  the  filtering  and  prediction  problems  have  been 
given  by  Kalman  and  Bucy*.  Solutions  to  these  latter  two  problems 
by  the  present  method  are  given  in  the  next  section. 


It  is  also  possible  to  find  the  covariance  matrix  of  errors  in  the 
smoothing  estimate  x(t)  by  corrsiierlng  small  variations  in  x,  x,  u,  u, 
z,  "z  in  Eqns.  (19)  through  (22): 


(34) 

d 

—  ’ 

Ox 

F 

gQg'^ 

ox 

+ 

G6u 

(35) 

•ar 

6X 

-fT 

6X 

(36)  6X(tj)  =  0 

(37)  6i(.^)  - 


Let 


(38) 


P^(t)  =  E(6^6x'^) 

<  “  E(656X^) 

P^^  (t)  ss  E(6X  6X^) 


♦Kalman,  R.  E. ,  and  Bucy,  R.  S. ,  ”New  Results  in  Linear  Filtering  and 
Prediction  Theory,”  Jour.  Basic  Engr.r  March  1961, 
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It  then  follows  immediately  from  Eqns.  (34)  to  (37)  and  the  relations 


(39)  E[fiu(t)6u(T)'^]  =  Q(t)6(t-T), 

(40)  E[6z(t)5z(T)'^J  =  R(t)6(t-T), 

(41)  E  [  6u(t)  6  z(t)^]  =  0, 

that  the  covariance  matrices  (38)  are  solutions  to 

(42)  P  =  jrp  +  PJ=^  +  S- 

F  ,  GSG^ 

-F^ 

GQG^,  0 

T _ 1 

0  ,  H^R 


where 

(43,44)  P  = 

T* 

^xx' 

.  = 

L  xx ' 

^xx_ 

The  boundary  conditions  for  (42)  are 


(45) 


:  -  Pxa(*o'Po  * 


Thus,  finding  the  covariance  matrices  is  also  a  two- point  botmdary 
value  problem. 


It  can  be  shown  that  the  correlation  matrix  of  errors  in  u(t)  is  given 

(46)  GQG^  =  GQG^  +  GQG^pJ^  +P^G(3g^  , 
where 

(47)  E  ( 5u(t)  6u(t)'^ ]  s  Q(t)  6  (t  -  t)  . 

Similarlyi 

(48)  H^R"^RR“^H  s  R ^  R"^H  , 

where 

(49)  EI6z(t)6z(T)'^]  ■  R(t)6(t-T)  , 
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FILTERING  FOR  LINEAR  SYSTEMS 

If  only  x(t^)  is  wanted^  the  two  point  boundary  value  problem  of 
Eqs.  (19)-  (22)  may  be  converted  to  an  initial  value  problem.  Referring 
to  Eq.  (31),  we  define  a  new  quantity 

(50)  y(t)  =  Xp{t)  -  *^(t)[*^(t)]"^Xp{t) 

Note  that 

(51)  y(t^)  =  x(tp 


Let  us  also  define  another  quantity 
(52)  Py{t)  = 

Differentiating  (52)  with  respect  to  t  and  using  the  relations 


(53) 


dt 


K 

—  T~ 

F  ,  GQG 

$ 

X 

X 

t.T — 1  T 

H  R  "h.  -F 

(54) 


W  -  ‘ 


It  is  straightforward  to  show  that 

(55)  a  FP^  +  P^F^  -  PyH^R'^MPy.  +  GQG^ 


(56) 


P  (t  ) 
y'  o' 


Differentiating  (50)  with  respect  to  t  and  using  (19),  (20),  (23),  (24),  (52), 
and  (55),  it  is  also  straightforward  to  show  that 


(57)  y  =  Fy  +  Gu  +  Py  H^r"\z  -  Hy  -  w) 

(58)  y(y  = 


Eqs,  (55)  through  (58)  are  precisely  those  obtained  earlier  by  Kalman  and 
Bucy  (Loc.  Git. ).  Fig.  1  shows  a  block  diagram  of  this  filter.  Note  that 
^(t)  rriay  be  precalculated  or  calculated  simultaneously  with  y(t). 
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The  solution  to  the  smoothing  problem  (linear)  of  Eqs.  (19)  through 
(22)  can  be  obtained  by  "filtering”  to  obtain  y(t£)  =  x(t£),  then  integrating 
Eqs,  (19)  and  (20)  backward  with  boundary  conditions 

(59, 60)  x(t^)  =  y(tf)  ;  X{t^)  =  0  . 

A  block  diagram  of  this  "smoother"  is  shown  in  Fig.  2. 

Similarly  it  can  be  shown  that 

461)  =  p^(,,)  . 

Hence,  the  solution  to  the  covariance  Eqs.  (42)-  (45)  for  the  smoothing 
problem  can  be  obtained  by  "filtering"  to  obtain  *  then 

integrating  Eqs.  (42)-  (44)  backward  with  boundary  conditions 

=  V*t'  • 

(63)  P^^(.j)  »  0  , 

(64)  P^^(,j)  =  0  . 

The  impulse  response  functions  (kernels)  for  the  linear  filter  are 
easily  found  using  the  equations  adjoint  to  equations  (19)  and  (20)  , 


(65) 

(66) 


ttT— -1 

-F* 

R  H 

-GQG^, 

F 

__ 

with  boundary  conditions, 

(67)  =  I  3  unit  matrix 

(68)  y*olPo*A‘y  = 


It  follows  immediately  that 


FIGURE  1.  OPTIMAL  FILTER  FOR  LINEAR  MULTIVARIABLE  SYSTEM 
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Due  to  evitable  conflicts  of  space  and  time,  this  paper 
appears  here  only  in  digest  form.  A  detailed  exposition  will 
appear  in  the  near  future  as  a  RIAS  repoit  [5]. 

The  principal  cont^ib^xtloal  of  this  paper  is  the  recogni¬ 
tion  that  Caratheodory’s  classification  of  regular  extremals  and 
the  theory  of  controllability  are  closely  related  topics. 

1.  All  notations  being  the  same  as  in  [1],  consider  a 

standard  variational  problem. of  the  following  type:  (i)  The 

initial  point  =  ((t^,  ^0^^  motion  to  be  optimized  is 

fixed;  (ii)  the  final  point  must  lie  on  the  surface  S^  in 

(^+n^-  dimensional  (t,  x) -space;  (iii)  the  motions  are  governed 

by  the  equation  dx/dt  =  f(t,  x,  u(t)),  (iv)  values  of  u(t) 

lie  in  the  set  U(t)  for  each  t;  (v)  the  quantity  to  be 

minimized  is  the  integral  of  function  L(t,  x(t),  u(t))  along 

the  motion  from  P  to  S, . 

o  1 

Uhder  these  circumstances,  the  problem  is  analyzed  [l] 
by  defining  first  a  pre -Hamiltonian  function 

(1,1)  H(t,  X,  p,  u)  =  L(t,  X,  u)  +  (p,  f  (t,  X,  u)). 
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Tbe  n-vector  p  Is  called  the  costate. 


PontiTagln's  theoreia  states  that  erezy  minimizing  motion 
Is  a  pseudo-extremal.  This  means  that  ^  x°(t)  Is  an  optimal 
motion,  generated  by  the  contral  fvmctlon  u°(t),  then  there 
exists  also  a  comotion  p°(t)  such  that 

(1.2)  H(t,  x°(t),  p°(t),  u)  i  H(t,  x°(t),  p°(t))  for  all  u  e  U(t) 

along  any  optimal  motion,  and  (x°(t),  p°(t))  are  solutions  of  the 
equations 


dx/dt  =  ^(t,  X,  p,  u°(t)) 

(1-3) 

dp/dt  =  ..  Hr(t,  X,  p,  u^(t)). 

Any  pair  (x^(t)^  P^(^))  which  satisfies  (2-3)  is  a  pseudo-extremal, 
rather  than  an  extremal^  hecause  in  general  one  cannot  prove  that 
such  motions  are  optimal  over  any  interval  of  time. 

A  pseudo-extremal  is  regular  if  (1.2)  holds  with  a  strict 
inequality  sign  for  all  u  ^  u°(t).  In  that  case,  subject  only  to 
mild  smoothness  conditions  on  the  set  U(t),  eqmtlons  (1.3)  nay 
be  replaced  by  an  equivalent  set,  which  however  does  not  involve 
u°(t), 

dx/dt  =  H°(t,  X,  p) 

(1.4) 

dp/dt  =  -  I^(t,  X,  p). 


where 


^Although  p°(t)  Is  not  uniquely  defined  by  x*^(t),  it  can  be  shown 
that  the  choice  of  p°(t)  is  immaterial  as  far  as  the  definition  of 
regularity  is  concerned. 
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(1-5) 


H°(t,  X,  p)  *  min  H(t,  x,  p,  u), 
u  €  U(t) 

is  the  true  (claBBlcal)  Hamiltonian  function  of  the  problem. 

(1.4)  are  the  well-known  Euler-Iagrange -Hamilton  equatlonB. 

✓ 

With  Caratheodory,  an  extr!emal  is  djefined  as  a  motion 
=  (x^(t),  p^(t))  which  minimizes  the  integral  /  L  dt  with 
respect  to  any  comparison  curve  that  ccnnects  two  points  and 

Pg  on  X  ,  provided  P^  and  P^  are  sufficiently  close.  This 
is  optimality  over  small  intervals  of  time. 

The  principal  theorem  of  the  classical  calpulus  of  varia¬ 
tions  then  asserts :  Every  regular  pseudo-extremal  is  an  extremal. 

In  applying  the  Hamiltonian  theory  to  the  problem  of  second 
variation,  several  authors  (Bryson,  Kelley,  etc.)  proceed  by  using 
equations  (1.2-3),  subject  to  a  side  condition  which  giiarantees  that 
the  first  variation  vanishes.  In  the  regular  casej  this  is  lannecessary 
if  the  formalism  of  (1.4-5)  is  used.  In  these  equations,  it  is  not 
even  necessary  to  know  what  the  variation  of  u  has  to  be,  since 
the  values  of  u  are  implicitly  determined  onpe  and  for  all  by  (l.5)« 

2.  Concentrating  attention  on  the  regular  case,  that  is, 
(1.4-5),  it  is  well  known  that  the  problem  of  second  variation  may 
be  studied  advan'^geously  by  considering  the  so-called  accessory 
(or  tJhcobl)  variational  problem,  which  has  a  qiiadratic  Hamiltonian 
function.  For  this  type  of  Hamiltonian,  the  variational  problem  has 
linear  character,  and  there  is  an  extensive  and  useftil  matbematical 
literature  (see  [1]  for  further  references.) 

To  be  more  specific,  let  =  (x^(t),  P°(t))  be  a  regular 
extremal.  If  we  consider  small  variations  about  that  is,  the 

behavior  of  |  =  x  -  and  ir  =  p  -  p°,  we  are  led  to 

4S/M  . 

(2.1) 

te/dt  •  ^ . 
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iriiexe  the  OBtrices  etc.,  are  evaluated  along  the  path 

and  axe  In  genezal  functions  of  time .  The  regularity  condition 
Is  equivalent  to  the  fact  that  the  matrix  condition  Is  non- 

positive  definite.  Tte  Hsmlltonian  function  associated  with 
(2.1)  is 


(2.2) 


IlSll  o  2<ir,  l^|>  +  lk||%. 
^  “pp 


If  ye  define 

^(t)  =  F(t),  H^p(t)  =  -G(t)G'(t),  ^(t)  =  Q(t), 

then  the  Hamiltonian  (2.2)  and  the  Euler  equations  (2.1)  are  pre¬ 
cisely  those  which  result  from  the  following  variational  problem; 
(l-ii)  It  is  required  to  transfer  the  point  (t^,  |^)  to  the 
surface  which  is  the  tangent  plane  to  at  the  point 

where  meets  S^;  (ili)  'the  eqmtions  of  motion  are 

(2.3)  ifi/dt  =  F(t)|  +  G(t)ji(t), 


(iv)  there  are  no  restrictions  on  values  of  |i(t);  (v)  the 

Lagrangian  is  defined  as 

(2.4)  2L(t,  I,  n)  =  ||6|||  +  yf. 

It  Is  then  easy  to  verify  that  the  Bamlltohlan  H°  corresponding 
to  (2.4)  Is  precisely  (2.2);  moreover 

(2.5)  u°  =  -  G*p. 

To  study  the  variational  problem  defined  by  (2. 3)  and 
(2.4),  It  Is  Important  to  Investigate  the  notion  of  controllability 
[2]  of  the  system  (2.3).  This  question,  which  concerns  the  accessory 
problem,  also  has  a  nattiral  Interpretation  for  the  original  variational 
problem. 
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3.  In  a  celebrated  paper  [3]  published  in  1935^  Caratheodoiy 
introdxaced  a  dassification  of  regular  extremals:  Take  a  given 
(regxilar)  extremal  Find  all  nearby  extremals  which  termin¬ 

ate  at  P^.  These  extremals  will  fill  out  a  "manifold”  of  points 
which  includes  Let  the  dimension  of  this  manifold  by 

n  +  1  -  q  (q  ^  0) .  Then  the  integer  q  is  the  Caratheodory  class 
of  the  extremal. 

Variational  problems  of  class  0  behave  as  the  conven¬ 
tional  variational  problems  without  side  conditions.  On  the  other 
hand,  all  regular  extremals  of  a  Mayer  problem  are  of  class  q  ^  1. 

Time  optimal  problems  are  often  of  class  q  =  n  -  1.  Thus  the 
Caratheodory  class  indicates  the  difficulty  of  a  variational  problem 
of  the  lagrange  type,  i.e.,  of  problems of  interest  in  optimal  control 
theory. 

Caratheodory  [5]  bas  shown  also  that  the  class  number  of 
a  regular  extremal  can  be  calculated  by  forming  certain  determinants 
related  to  the  variational  equations  (2.1).  It  turns  out  that  his 
technique  is  identical  with  methods  used  (independently)  by  the 
writer  for  the  study  of  controllability  and  observability  questions 
in  linear  systems  [2].  The  situations  to  which  this  technique  is 
applied  are  different,  but  it  is  possible  to  connect  the  two  notions. 
The  following  result  is  typical: 

A  (regular)  extreaaal  is  of  class  q  =  0  if  and  only  if  its 
variational  equations  are  completely  controllable. 

The  proof  is  essentially  the  same  as  was  given  in  [4]  for  a 
slightly  different  problem. 

^  In  general,  n  ••  q  =  dimension  of  the  space  of  controllable 
states  of  (2.3)* 

A  trivial,  but  practically  necessa^,  extension  of  the 
Caratheodory  classification  is  the  followjLng:;  Given  a  regular  extremal 
terminating  on  the  surface  let  n  +  1  -  qi(Oj^)  be  the 

dimension  of  the  space  filled  oid;  by  liearby  extremals  which  also 
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terminate  on  This  defines  the  class  q(Sj^)  of  rela¬ 

tive  to  Sj^.  Since  it  is  easier  to  "hit”  a  surface  than  a 
pointy  in  general  \re  will  have  that  q(S^)  2:  q.  This  concept  has 
not  been  investigated  so  far. 

4.  The  technique  outlined  above  can  be  applied  piSDfit- 
ably  to  many  problems  of  trajectory  optimization.  In  fact,  the 
study  of  the  "class"  often  coincides  with  the  study  of  the  Jacobi 
determinant,  which  is  necessary  to  locate  conjugate  points,  Ey 
concentrating  attention  on  the  geometric  properties  of  nearby 
extremals,  one  obtains  a  precise  and  intuitively  appealing  char¬ 
acterization  of  optimal  trajectories. 

Full  details  will  be  found  in  [5]. 
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ABSTRACT 

A  method  is  suggested  for  computing  optimal  controls.  This  method  is 
based  on  the  Pontryagin  maximum  principle,  and  consists  in  periodically 
computing  the  appropriate  initial  conditions  of  the  linearized  adjoint  system 
as  a  function  of  the  state  of  the  system.  The  adjoint  system  is  then  used  to 
determine  the  optimal  control.  Detailed  computational  procedures  -  using 
a  steepest  ascent  method  of  successive  approximations  -  are  developed  for 
some  specific  optimal  problems  including  the  ”bang-bang**  time -optimal 
and  minimum  control  energy  problems. 

INTRODUCTION 

One  of  the  difficvdties  which  arises  in  the  implementation  of  optimal 
control  systems  is  the  complex  control  law  that  is  generally  required.  In 
conventional  control  systems  the  controllers  are  made  to  be  simple  functions 
of  the  output  (or  state)  of  the  object  being  controlled  (the  so-called  plant). 
However,  if  the  plant  is  to  be  controlled  optimally,  the  relation  existing  at 
any  time  between  the  state  of  the  plant  and  the  value  of  the  optimal  control  is 
both  difficult  to  precompute  and  difficult  to  implement  in  an  actual  system. 

For  certain  types  of  control  systems,  and  certain  optimality  criteria, 
methods  have  been  described  for  precomputing  an  optimal  control  (References 
1  through  4).  In  each  of  these  methods,  the  entire  optimal  control  function 
can  be  computed  once  the  initial  and  desired  terminal  states  are  specified. 

In  principle,  a  system  using  these  methods  would  operate  as  follows:  the 
initial  state  of  the  system  is  measured  and  fed  into  a  computer  which  calcu¬ 
lates  the  time  history  of  the  optimal  control,  stores  the  latter,  and  drives 
the  controller  to  follow  the  stored  program.  Such  systems,  once  they  begin 
to  operate,  are,  of  course,  **open-loop";  i.e.  ,  no  corrections  are  made  to 
the  control  program  based  on  measurements  of  the  actual  state  of  the  plant. 
However,  "feedback”  can  be  introduced  by  periodically  updating  the  control 
program  by  recomputing  the  optimal  control  at  any  time,  considering  the 
state  of  the  plant  at  that  time  to  be  an  "initial  condition.” 

By  suitably  choosing  our  computational  method,  we  shall  describe  one 
such  procedure  which  appears  to  be  particularly  suitable  to  such  "updating? 
Our  method,  which  is  based  on  the  Pontryagin  maximum  principle,  can  in 
principle  be  applied  to  a  wide  class  of  optimal  control  systems.  However, 
computational  algorithms  have  so  far  been  developed  for  only  certain  parti¬ 
cular  types  of  systems,  which  do,  however,  include  many  important  special 
cases.  These  algorithms  are  described  in  Section  III. 
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GENERAL  DESCRIPTION  OF  THE  SYNTHESIS  PROCEDURE 

We  confine  ourselves  to  control  systems  whose  behavior  can  be  described 
by  a  system  of  ordinary  differential  equations  which  have  the  form 


=  f.  (x^  ,  .  .  .  >  ^  •  I  u^  ,  t)  ,  i  =  1  ,  .  .  .  ,  n  .  (1) 

The  describe  the  state  of  the  system,  and  the  uj  describe  the  state  of 
the  controllers.  We  assume  the  f^  to  be  continuous  in  all  their  arguments, 
and  to  be  continuously  differentiable  with  respect  to  the  xj  and  t.  If  we 
denote  by  x  the  vector  (xj,  .  , .  ,  x^^),  by  u  the  vector  (uf ,  .  .  .  ,  Uj.),  arid 
by  f  the  vector  (ff,  .  .  .  ,  fn) »  Equations  (1)  can  be  written  in  vector  form  as 


X  =  f  (x,  u,  t) 


(2) 


The  control  vector  u  may,  of  course,  depend  on  the  time,  but  we  shall 
require  u(t)  to  be  piecewise  continuous  for  reasons  of  physical  realizability. 
Finally,  we  shall  suppose  that  the  controllers  are  constrained  and  that  this 
constraint  can  be  described  by  the  requirement  that  u(t)€n  for  all  t,  where 
n  is  a  set  in  r- space  which  is  independent  of  t  and  x.  We  assume  that  there 
are  no  explicit  constraints  on  the  values  of  x. 


We  suppose  that  the  aim  of  the  control  process  is  to  transfer  the  system 
from  one  given  state  to  another.  That  is,  we  are  given  two  values  x^  and 
x^  of  the  vector  x,  and  an  initial  time,  to;  we  wish  to  find  an  admissible 
control  (i-e.  ,  a  piecewise  continuous  function  whose  range  is  in  Q)  u{t)  such 
that  the  solution  of  Equation  (Z)  with  u  =  u(t),  x(to)  =  x®  takes  on  the  value 
x^  at  some  time  tp  >  to-  The  terminal  time  tp  may  be  fixed  or  free.  The 
case  where  the  target  position  depends  on  the  time:  x^  =  x^(t),  can  be  put 
into  the  same  form  by  defining  a  new  variable  x  =  x(t)  -  x^t)  (assuming  that 
x^t)  is  twice  continuously  differentiable).  Also,  the  aim  of  the  process  may 
be  to  attain  a  set  rather  than  a  point.  In  many  cases  ,  the  treatment  of  the 
problem  is  then  only  slightly  more  complicated  than  if  a  point  is  to  be 
obtained  (e.  g.  ,  see  Reference  4,  Section  IV),  and  we  shall  confine  ourselves 
to  point  targets. 


Let  us  suppose  that  the  optimality  criterion  is  given  by  an  integral  of 
the  form: 


(x  (t)  ,  u(t),  t)  dt 


(3) 


where  the  function  Iq  satisfies  the  same  conditions  as  f  j ,  .  .  .  ,  fy^.  Thus, 
our  airn  is  to  find  an  admissible  control  u{t)  which  "trarisfers**  the  control 
system  given  by  Equation  (Z)  fronfi  a  given  state  x^  at  the  time  t  =  to  to 
another  state  x^  at  the  time  tp  (which  may  be  either  fixed  or  free)  in  such 
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a  way  as  to  minimize  the  integral  (3)«  Let  us  assume  that  such  an  optimal 
control  exists.  (The  existence  problem  is  itself  of  interest.  For  discussions 
thereof  see  Reference  5,  §  19  and  References  6  through  9-)  Then,  according 
to  the  Pontryagin  maximum  principle  (Reference  5,  Chapters  1  and  II), 
there  exists  a  nontrivial  solution  ♦(t)  =  tl(t)*  •  •  •  system 

of  equations; 


n  af.  (x*(t),  u*(t),  t) 

E  -  .  ^.,i  =  l,...,n  , 

j=0  ^^i  J 


const.^0  , 


(4) 


such  that 

^  *:(!)  f.(x^(t),  u*(t),  t)  =  max  Z  t.(t)  f.  (x*(t),  u,  t)  (5) 

i=0  ucfl  i=0 


at  all  points  of  continuity  of  u  (t)  -  where  we  have  denoted  the  optimal  control 
and  optimal  trajectory  by  u*(t)  and  x*(t),  respectively. 

Thus,  if  the  initial  conditions  for  the  system  (4)  are  known,  it  is  possible 
to  simultaneously  solve  Equations  (1),  (4),  and  (5)  (provided  that  relation  (5) 
determines  u*(t)  \miquely  for  t  in  a  dense  subset  of  (to»tp)),  and  thereby 
compute  the  optimal  control  u*(t). 

Let  the  point  be  fixed.  Suppose  that  an  optimal  control  exijsts  for 
(xO^,  t^)  in  some  region  R  in  (x,t)  space,  and  that  for  each  initial  value  in 
R  this  optimal  control  is  unique  (if  we  disregard  values  at  points  of  discon¬ 
tinuity).  Then,  for  each  such  initial  value,  there  is  a  solution  #(t)  of 
Equations  (4) ,  with  initial  value  which  determines  the  optimal  control 
u^(t;  x®,  t^)  through  relation  (5).  The  corresponding  solution  f  (t)  is  not 
unique  (it  certainly  can  be  multiplied  by  an  arbitrary  positive  constant  with¬ 
out  affecting  the  maximum  in  Equation  (5))..  Let  us  denote  by  H(xO;  t^) 
the  set  of  all  (n+l)-vectors  which  have  the  following  property:  the 
control  u*(t),  determined  by  J^uatipn  (5)  with  f  (t)  the  solution  of  Equation 
(4)  satisfying  the  initial  condition  ♦(tQ)  =  optimally  transfers  the  control 

system  from  x^  at  the  time  tp  to  the  state  x^  . 

4^  4i-  . 

If  (x® ,  t^)  is  a  fixed  point  in  R,  x  (t)  and  u  (t)  are  the  corresponding 
optimal  tr^ectory  and  control,  and  u*(t)  is  determined  by  a  particular 
solution  i  u)  of  Equations  (4)  (so  that  i  '  (tp)  €H(x®;  tp))  >  it  is  clear  that 
and 

j*(F)  (H  (x  (t  );  T)  (6) 

for  any  t€(t  #tp)  (since  the  portion  of  the  control  process  for  t>  t  must 
also  be  optimalf. 
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We  are  now  in  position  to  describe 
At  the  initial  time  t, 

Depending  on  the  target  state 


*0  of  the  set  H(xO;  to)  is  found. 


the  suggested  synthesis  procedure. 

xO  is  measured, 
and  the  integral  functional  J,  one  member 
Then,  for  a  short  time  interval.  Equations 


Q,  the  initia^  state  of  the  system 


(1),  (4),  (5)  are  solved  simultaneously  -  in  real  time  -  and  the  optimal  control 


u^(t)  is  computed.  At  the  end  of  this  interval,  say  at  the  time  tp  the  state 
of  the  system  x(tt)  is  measured,  and  a  member  i|f^  of  the  set  H  (x(t^);  t^) 
is  found.  The  procedure  is  then  repeated  until  the  end  of  the  process.  The 
system  thus  operates  similarly  to  a  sampled-data  feedback  control  system.. 


The  principal  difficulty  in  applying  the  above  method  lies  in  finding  a 
member  of  the  set  H(x;  t).  For  certain  particular  optimal  processes,  as 
we  shall  describe  below  in  Section  III,  a  successive  approximation  method 
can  be  employed.  Such  a  method  is  particularly  attractive  for  the  following 
Teason.  If  €H(x(ti);  t^)  is  the  initial  value  for  system  (4)  in  the  interval 
(fi»ti+l)>  and  if  the  corresponding  solution  of  Equations  (4)is  \!f>(t),  then, 
under  ideal  operation,  tM^i+l)  €H(x(ti^^);  ti^-i)  because  of  relation  (6). 

Thus,  if  H(x;  t)  is  continuous  at  (xfti-i-i),  t^^[)  in  an  obvious  sense, 

should  be  an  excellent  approximation  to  a  member  of  H(x{ti4.i);  ti+i) 
even  in  the  presence  of  disturbances. 


During  an  interval  (t^,  q+i)  Equations  (1),  (4),  (5)  should  be  relatively 
easy  to  compute  in  "real  time."  For  example.  Equations  (1)  and  (4)  may  be 
integrated  on  an  analog  computer.  We  assume,  of  course,  that  relation  (5) 
uniquely  determines  u*(t).  In  many  problems,  it  turns  out  that  the  maximum 
in  this  relation  is  a  relatively  simple  function  of  x*,T)r,  and  t.  For  example, 
this  is  true  in  the  cases  described  in  Section  III  below.  In  other  problems, 
the  maximum  may  have  ^o  be  computed  by  more  sophisticated  techniques 
such  as  nonlinear  programming. 

SPECIFIC  COMPUTATIONAL  METHODS 

In  this  section  we  shall  describe  successive  approximation  procedures 
for  finding  members  of  the  sets  H(x;t)  in  some  specific  optimization 
problems.  We  shall  omit  the  derivations  of  these  methods  since  they  have 
been  presented  elsewhere  (see  References  3  and  10). 

First,  consider  the  well-known  "bang -bang"  regulator  problem^  Let 
Equations  (1)  have  the  form 


X  =  A(t)  X  +  B(t)  u  (7) 

where  A{t)  is  an  nX  n  matrix,  B(t)  is  an  nXr  matrix,  and  both  A  and  B 
depend  continuously  on  the  time,  t.  Let  ^  be  the  unit  cube  defined  by 
I'^il  ^  i  =  ,  r;  let  the  target  x^  be  the  origin:  x^  =  0;  and  let 

f^  =  1.  Thus,  J  =  tp- tQ,  and  the  minimization  is  with  respect  to  time 
(clearly,  tp  is  freej. 

Let  X(t)  be  the  nx  n  matrix  solution  of  the  system 

X(t)  =  A{t)  X(t)  ,  identity  . 
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Relations  (4)  and  (5)  ^re  now  equivalent  to 


n 


♦  s  =  -  2  a  (t)  *  (t)  ,  i  =  1  ,  ....  n  , 


j=l 


' j 


u*  (t)  =  ♦j(t)  bjj(t)j  .  j  =  1  .  . .  .  .  r  . 


(8) 


(9) 


where  A  -  and  B  =  Let  us  assume  that  the  zeros  of  Sf=iii(t)bij(t) 

are  isolated  lor  every  nontrivial  solution  of  Equations  (8)  (and  each 
j  =  1, .  .  .  ,  r),  so  that  Equation  (9)  uniquely  determines  a  piecewise  continuous 
control. 


For  a  given  x®  and  Iq  we  wish  to  find  an  h- vector  such  that 

relation  (9)  -  with  (f  j(t),  .  .  .  ,  =  ^(t)  the  solution  of  Equations  (8) 

with  t{tQ)  =  -  defines  the  time -optimal  control  which  transfers  the 

system  given  by  Equation  (7)  from  at  the  time  to  the  origin.  The 
set  of  all  such  vectors  will  be  denoted  by  h(x®;  tp)* 


We  define  a  function  F(^;  x  ,  t  ),  where  ^  is  an  n-vector,  such  that 
F  takes  on  its  maximum  value  when,  and  only  when,  r|  €  h(x®;tQ).  Namely, 


let 


rt 


f(t,  Tl;  X  ,  t^)  = 


.  X  ^(s)  B(8)  u(8;T])  ds  +  T]  •  x° 


where  u(s;Ti)  is  defined  by 

■  •  I 

Uj(t;  Tl)  =  sign  (Tl  •  X  *{t)  b^(t))  , 

being  the  j-th  column  of  B.  Then,  it  is  easy  to  see  that  f  is  a  strictly 
increasing  function  of  t  (for  fixed  T)  /  0).  We  shall  restrict  ourselves  to 
those  T|  for  which  T]  *  x^  <  0*  so  that  f(0,  x^,  t^)  <  6.  It  can  be  shown 

that  if  t*  is  the  minimum  time  in  which  the  origin  can  be  attained,  then 
f(t*,  Tl;  x^,  to)  >  0,  where  equality  holds  if,  and  only  if,  T]  €  h(x°;  Iq).  Thus, 
if  we  define  F(ti;  x^,  t^)  as  the  solution  of  the  equation 


f(F(Ti;  x^,  t^),  Ti;  x°)  =  0  . 

F  has  the  desired  properties,  the  maximum  of  F  being  the  minimum 
transfer  time  t  . 

Finally,  the  maximum  of  F  can  be  computed  directly  by  the  method 
of-steepe St  ascent,  since  the  gradient  of  F(ti|  is  proportiOhal  to 
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F(ti) 


X  ^(s)  B(s)  u(s;  Ti)  ds  +  x° 


Also,  the  gradient  is  different  from  zero  if  n  4  hi  so  that  F  has  no  local 
maxima  away  from  h. 

Generalizations  of  this  problem,  wherein  0  is  replaced  by  an  arbitrary 
convex,  compact  set,  and  the  origin  is  replaced  by  an  arbitrary  target  point 
X*,  are  described  in  References  3  and  iO. 

Computer  solutions  for  the  problem  described  above,  using  both  analog 
and  digital  computers,  have  been  described  by  Paiewonsky  and  Williamson 
(see  Reference  11). 

As  a  second  example,  consider  a  control  system  whose  behavior  is 
again  described  by  EqiTlifiOn  (7),  where  Q  is  as  before,  the  target  x^  is 
arbitrary,  the  terminal  time  tp  is  fixed,  and  is  given  by 


fo(u) 


r 


E 

j=l 


J 


(10) 


1 


and  Xj  ^  0 


(but  not  all  zero)  being  constants. 


Then, 


and  the  physical  meaning  of  J,  particxilarly  if  p  -  1  or  2,  is  clear. 
Relations  (4)  are  again  equivalent  to  Equations  (8).  With  little  loss  of 
generality,  assume  that  in  Equation  (4)  is  equal  to  -1.  Then, 
relation  (5)  takes  the  form 


u:(t)  =  -( 


|C.(t)/pX.|‘/<P-^^  sign  Cj(t)  if  |£j(t)|  S  pXj  and  i  0  . 
sign  £j(t)  ,  if  |Cj(t)|  >pX.  and  V  0  , 


(11) 


sign  Cj(t)  .  if  Xj  =  0  , 


for  p  >  1 ,  and 


u*(t)  = 


'o  if  fc.(t)|<x 

•J,  J 

sign  Cj  if  j£j(t)|  >  Xj 


(12) 
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if  p  =  1.  Here,  Cj(t)  =2^=1 

Cj(t)  arc  isolated  if  p>  1  and  Xj  =  0, 

are  isolated  if  p  =  1,  for  every  j  =  1,  . 


We  assume  that  the  zeros  of 
and  that  the  zeros  of  Cj(^)  ^ 

.  •  .  I  r,  for  any  nonzero  solution^ 


of  Equations  (8). 


We  define  the  set  of  n-vectors  h(x^;  t^)  in  a  manner  analogous  to 
before,  and  we  define  a  function  g(il;  tQ)  which  takes  on  its  maximum 
value  when,  and  only  when,  Tl€h{x®;  t^).  Namely,  let 


gh;  x°,  t^) 


x^- 


jjT  X“  ^  (t)  B(t)  u(t;  Tl)  +  (u(t;  n))]  dt  -  tt  x' 


where  u(t;il)  is  given  by  Equation  (5)  (or  Equations  fit)  or  (12))  with 
t|(t),  .  .  .  ,  equal  to  the  components  of  T\  •  X“^(t).  (It  is  well-known 

that  the  latter  vector  is  the  solution  of  Equations  (8)  satisfying  the  initial 
values  t^ft^)  =T1^.)  Furthermore,  the  maximum  of  g  can  be  computed 
by  the  method  of  steepest  ascent  using  the  fact  that 


gradg(Tl) 


X“^{tp)  x^  - 


(t)  B(t)  dt 


and  noting  that  g  has  no  local  maxima  away  from  h. 

The  relations  above  are  derived  in  Reference  10,  wherein  more  general 

functions  f  are  also  treated, 
o 

The  case  where  f  is  defined  by 
o  ^ 

foM  =  |u(t)||  . 

which  arises  in  minimum  fuel  guidance  problems,  is  described  extensively 
in  Reference  12. 

As  a  final  example,  consider  again  the  case  where  f^  is  defined  by 
Equation  (10),  where  now  p>  1,  every  is  positive,  and  0  is  the  entire 
r-dimensional  space  (so  that  there  are  no*^ restrictions  on  the  values  of  u). 
Then,  the  treatment  differs  from  the  one  above  only  in  that  Equation  (11)  is 
replaced  by 

U*(t)  =  jCj(t)  /pX.j|  ^  •  sign  Cj(t)  . 

and  the  definition  of  ^(t;  Tl)  is  changed  accordingly. 

If  variations  of  the  above  examples  are  considered,  it  is  often  easy 
to  construct  functions  haying  properties  similar  to  those  of  F  and  g  which 
lead  to  a  successive  approximation  method  of  finding  the  appropriate  initial 
conditions  to  Equations  (4). 
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INTRODUCTION 


Let  us  consider  an  example  which  illustrates  the  concepts 
we  wish  to  discuss. 

The  small  amplitude  pitch  motions  of  many  aircraft  can  be 
summarized  approximately  by  solutions  of  the  equations 

(a)  a  -  k^  a  -  0  =  k2  6  (Flight  Path  equation) 

(b)  0  -  k^  5  -  k^^  a  -  a  =  kg  5  (Pitch  equation) 

(c)  6  =  u  (Servo  equation) 

Here 


a  ~  aircraft  fuselage  reference  angle  of  attack 
0  pitch  attitude  change 
6  ~  elevator  deflection 

u  ~  control  variable,  and  k^^  =  constants. 


We  shall  assume,  as  is  natural. 


u_<  1, 


Consider  the  relationship  between  u  and  0,  Letting  x=0 
and  eliminating  a,  and  d  from  (a),  (b),  and  the  derivative  of 
(b)  gives 

kgk^ 


(d)  X  +(-k2-k5-kj^)x 


+1  6  + 


Differentiating  (d)  and  using  (c)  to  eliminate  6  gives  an 
equation  of  the  form 


••  ••  , 

(e)  X  +  euc  +  bx 


cu  +  du 


+The  results  reported  here  were  achieved  at  the  Minneapolls- 
Hoheywell  Regulator  Company,  \inder  sponsorship  of  NASA 
Contract  NASr  27. 
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The  transfer  function  for  ^  is  ^ -  ,  from  which 

s(s*+as+b) 

the  term  numerator  dynamics  stems. 

If  we  consider  (e)  to  be  the  equation  of  motion  we  run 
into  certain  technical  difficulties.  For  example,  how  one  de¬ 
fines  a  solution  if  the  control  u  =  u(t)  was  composed  of  a 
series  of  step  functions.  There  are  now  standard  procedures 
for  transforming  equations  of  the  form  of  (e)  into  a  system  of 
equations  in  which  the  derivatives  of  the  control  variable  do 
not  appear.  (See  for  example,  Lanlng  and  Battln,  "Random 
Processes  in  Automatic  Control",  p.  19I,  or  Just  reverse  the 
above  procedure.)  We  prefer  to  work  with  the  system  (a)  -  (c). 

A  typical  control  problem  is  to  reduce  the  pitch  rate, 
d  =  X,  to  zero  in  finite  time  and  In  such  a  manner  that  it  will 
remain  zero  if  no  additional  disturbances  appear.  Actually  we 
should  impose  certain  conditions  on  the  other  variables,  for 
example,  require  [&  £  e  (the  bounded  phase  coordinate  problem). 

In  section  2  we  consider  the  various  modes  of  control. 

It  is  shown  that  most  optimal  control  problems  are  included  in 
the  problem  of  controlling  only  one  state  vector  component 
after  choosing  a  suitable  coordinate  system.  We  have  shown 
that  such  problems  arise  in  a  natural  way  when  considering 
system  transfer  functions  with  numerator  dynamics.  Section  3 
Is  devoted  to  necessary  and  sufficient  conditions  for  time 
optimal  control.  Here  we  show  that  control  of  one  state 
variable  is  equivalent  to  controlling  to  a  closed  convex  set 
of  possible  states. 

Section  4  is  devoted  to  considerations  of  constructing 
the  time  optimal  feedback  controller. 

MODES  OF  CONTROL 

Consider  a  physical  system  as  represented  by  the  equations^ 
x^  =  aj  x*^  +  b^u;  1,  J  =  l,2,...,nj  (l) 

or  in  vector  form 

X  =  Ax  +  bu 

here  X  is  the  system  state  and  u  the  control  variable. 

We  wish  to  determine  a  bounded  scalar  function  u  =  u(x) 
such  that  some  or  all  of  the  solution  components  x^(t),  i  =1, 
2,. .. ,n  of  (ij  behave  in  a  desired  manner  on  the  finite  time 
interval  0,  tT  . 

^  Repeated  index  indicates  sxiramation  with  respect  to  that  index. 
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We  shall  adhere  to  the  following  definition: 

Multiple  component  control  is  defined  as  control  of  less  than 
n  of  the  state  variables  by  bringing  them  to  zero  in  finite 
time  and  holding  them  at  zero  thereafter. 

Single  component  control  is  defined  as  control  of  a  single 
state  variable  by  bringing  this  variable  to  zero  in  finite 
time  and  holding  it  zero  thereafter. 

It  is  seen  that  single  component  control  is  a  special 
case  of  multiple  component  control.  Time-optimal  multiple 
component  control  is  defined  in  the  obvious  way  when  u(t) 
is  bounded;  namely,  bring  the  state  variables  to  be  controlled 
to  zero  in  the  minimum  time  such  that  they  may  be  held  at  zeTO 
thereafter  with  u(t)  satisfying  the  bounding  constraint.  Time 
optimal  single  component  control  is  discussed  in  section  3. 

It  is  now  our  purpose  to  show  that  multiple  component 
control  can  be  accomplished  by  single  component  control. 

As  in  jlj  suppose  that  for  a  system  described  by  (l)  it 
is  desired  to  control  the  variables  xl,  x^,..  1  £  m  ^  n. 

If  m  =  1,  the  problem  is  the  single  component  problem  of 
controlling  xl.  Thus,  assume  m  >  1  and  for  convenience 
Introduce  the  following  notation: 


Then  (l)  becomes 
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(2) 


V-  =  +  A® 

i®  =  a3  +  P®u 

and  it  Is  desired  that  be  controlled. 


1  1  •  T 

Holding  4  zero  requires  that  =  0,  in  which  case 

(2)  becomes 

0  =  A® 

I®  =  +  p®u 


If  =  0  and  A®  =  0,  it  is  impossible  to  control 
because  the  first  equation  of  (2)  becones  =  A^  and  the 
proposed  problem  is  of  no  Interest,  If  pi  =  0  and  A®  y  0, 
it  may  be  assumed  v/ithout  loss  of  generality  that  the  first  row 
of  A®  contains  a  non-zero  element.  Then  the  state  variable 


—  2  a^ 

y  -  jSnvfl  ^ 


is  necessarily  controlled  when  4^  is  controlled.  Thus,  the 
original  problem  can  be  reformulated  so  that  it  becomes  a 
problem  of  controlling  m+1  state  variables,  l,e,  xl,x® , , , , 
and  ynt*-l. 


Hence,  consider  the  case  when  y  0  and  assume  without 
loss  of  generality  that  b"^  /  0,  Making  a  transformation 


0 

y  =  Sx,  vfhere  S  = 

0  l’ 
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equations  (2)  become 

a2  52  ^  s^p^u 

(4) 

^  =  A3(s^r^  +  a"^  +  p^u 

S^  was  chosen  so  that 

-  To  I 


If  the  kth  row,  k  <  m,  of  A®  has  non-zero  elements, 
the  control  of  Implies  the  control  of  the  kth  component 
of  A^  and  the  original  problem  may  be  reformulated 
so  that  it  becomes  a  problem  of  controlling  m  +  1  state 
variables. 


Prom  the  foregoing  considerations,  it  is  clear  that  if 
it  is  possible  to  control  only,  then  (4)  may  be  obtained 
with 


0 

6 

• 

• 

S^A^= 

0  ... 

•'  . . 

.  • 

• 

.  0 

• 

S^A^(S^)"^= 

al(l) 

• 

• 

• 

m 

• 

• 

0 

bm 

0 

Ml) 

^flH-l  •  • 

0 

.  a”(l) 

n 

Ml) 

•  • 

.  a®tl) 

m 

Now  if  each  a^^  ^=0  with  1  <  m,  the  components  y^,  y^, ...y™" 
of  ?!  are  not  controllable}  hence  and  consequently  4^,  are 
not  controllable.  Thus  it  may  be  assumed  without  loss  of 
generality  that  0,  Define 
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and 

Then 


1(2) 


denote  the  IJth  element  of  [s^l  [s®]  by  a4 

0,  1  <  m  -  1  and  a5["^^^^=  0.  Now  If 

each  a^if '=  0  with  1  <  m  -  1,  the  first  m  -  2  components  of 

S®  are  not  controllable  and  hance  4^  Is  not  controllable. 
Thus  it  may  be  assumed  without  loss  of  generality  that 

j^m-2  (2  Q  ^  Define 

m-i  I —  " 

jjni-2(2)  Q  •  0 


m-1 


^“2(2)  .  0 


*m-l 


'  >-2(2) 

^m-1 


_a2(2) 

7m*l 

->-3(2) 

^m-1 
1 


0 

0 

• 

0 

0 

1 


0 

0 

• 

0 

0 

0 

1 


Continuing  this  process,  S*  =  s"'"^  s"'”^  .  .  .  S®  Is 
determined  so  that 


S*A^(S*)”^  has  the  form 


“1 

“2 

0 

— 

0 

“! 

a® 

2 

“1  -- 

—  - 

— - 

0 

„m-l 

“1 

MM 

—  — 

— 

^m-l 

CL 

m 

[“1 

— 

- - 

MM 

“m  _ 

with  ^  0;  1 

|> 

Letting  z  = 


l,2,...,m-l.  S*A®  =  S^A®  and  S* 


0 


rn-m 


X  and  C3  = 


F-il 

„IIH-1 

z 

Z 

i*. 

• 

• 

> 

•n 

z 

z 

(2)  becomes 


S*  A^(S*)“^  43  +  s*  ^2  +  sVu 

a3(s*)“^  43  +  A^  4^  +  p®u 


(5) 
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rom  (5)i 

.k+l 


z 


a 


k+lL 


2 

J=1  J 


Ic  ®  ni“l 


With  the  aid  of  this  recursive  relation,  it  is  possible 
to  obtain  as  a  linear  combination  of  z^  its  first  k-1 
derivatives  for  k  £  m,  ThuSi  controlling  z^  accomplishes 
control  of  $1. 

NECESSARY  AND  SUFFICIENT  CONDITIONS  FOR  TIME  OPTIMAL  CONTROL 


In  section  2  it  was  shown  that  any  multiple-component 
time-optimal  control  problem  for  a  linear  system  with  a  single 
control  variable  is  equivalent  to  a  single -component  time 
optimal  control  problem.  For  this  reason,  only  single- 
component  control  problems  need  be  considered. 

Suppose  x^(t)  is  the  variable  which  we  wish  to  bring  to 
zero  and  hold  at  zero  in  minlraiun  time  T  wlth(u|<  1.  If  x^(t) 
is  to  be  zero  for  t  >  T  then  <^(t)  =  0  for  t  >  T.  Thus 
aj  xJ(t)  +  bl  u(t)  =  0  for  t  >  T. 

Let  G  be  the  set  of  points  Xj^q,  X2o»...>XnO  Euclidean 
n-space  such  that  the  solution  of  1)  with  u^(07  =  xio  =  0, 
xJ(o)  =  x-io  satisfies  al  x«J(t)  +  b^u(t)  =  0  for  all  t  >.  0  for 
some  u(t)^with |u(t) I <  1.  .  . 

.1  -ai  x'>(t) 

Assxime  b  A  0;  thus  we  can  solve  for  u(t)  =  — ^ — 

b-^ 

and  substitute  this  result  into  equation  (l)  to  obtain  the 
homogeneous  equation 


x^  =  a^  x*^ - ^  x^y  i  -  2,3>.. .,n.  (h) 

^  b-^ 

A  point  p  =  (xio*  ^  solution  x^Ct) 

of  the  homogeneous  equation  satisfies 


*  If  hi  =0,  which  occurs  frequently,  we  obtain  the  condition 
that  aj  xJ-0  all  t  >  0  which  implies 

d(aixj) 

In  this  manner  we  eventually  obtain  a  condition  for  the  control 
as  a  function  of  the  state  variable  eind  reach  the  same  con¬ 
clusion  about  optimum  control  to  the  set  0. 
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\(t)  x^^Q 


<  1  for  all  t  ^  0, 


here  the  matrix  (9^(t))  is  the  fiiadamental  solution  matrix  of 
the  homogeneous  equation  (h). 

Let  J(t),  t  >  0,  be  the  set  of  points  (x,,.  =  0,  ,..x_^) 

such  that 

" 

thus  Q  =  /^  J(t),  which  is  thereby  closed,  convex  and  easily 
te  [o,Qo) 

shown  to  be  nonempty. 

We  conclude  that  control  of  one  component  of  the  state 
vector,  is  equivalent  to  control  of  the  state  vector  to  the 
closed,  convex  set  Gcr^i, 

Consideration  is  now  given  the  following  problem:  construct 
u  =  u(t),  with  |uK  1,  such  that  x(t)  -  (xl(t)  ,  x®  (t ) , , , ,  ,x^(t) ) 
moves  fixjm  x(o)  =  Xq  to  an  intersection  with  the  closed  convex 
target  set  G  in  minimum  time  T. 

For  the  remaining  discussion  assume  the  system  l)  is 
normal,  that  is,  the  vectors  b,  Ab, , . .  are  linearly 

independent. 

If  l)  is  normal  it  is  easily  proved  [2]  that  if  u(t)  is 
an  optimal  control  which  steers  x(t)  from  x©  to  G  it  is  the 
only  optimum  control  (in  the  sense  of  being  equal  almost  every¬ 
where)  . 

We  call  u(t)  an  extremal  control  if 
u(t)  =  sgn  e"^^  b] 

and  T)Qe  is  an  Interior  normal  to  G  at  the  point  x(t),  when¬ 
ever  the  response  x(t)  is  a  point  of  the  boundary  of  G.  Here 
Tjo  is  a  row  vector  of  unit  length. 

It  is  well  known  [2j  that  if  u(t)  is  an  extremal  control 
for  the  normal  system  l)  which  steers  x(t)  from  Xq  to  G  and 
G  is  such  that  a  nonextreraal  control  exists  so  that  each 
solution  of  l)  which  intersects  G  remains  thereafter  in  G, 
then  u(t)  is  the  unique  optimal  control. 

This  is  in  effect  a  necessary  and  sufficient  condition, 
because  optimal  control  is  necessarily  extremal  control. 
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SYNTHESIS  OP  TIME  OPTIMUM  FEEDBACK  CONTROL 

It  is  now  our  purpose  to  show  that  knowledge  of  the 
opotlmum  control  u(t)  for  various  initial  states  vjill  enable 
once  to  construct  the  feedback  controller  u  =  u(x(t)).  Before 
pr*?oceedlng  with  this  consider  the  question  of  existence  of 
ti—ine  optimal  control. 

It  has  been  demonstrated  [3]  that  if  there  exists  even  one 
al—lowable  control  which  steers  x(t)  from  Xq  to  G  for  the  nomal 
sjr^stem  l)  there  will  exist  an  optimiim  control.  Obviously  if 
thee  normal  system  1)  Is  stable  (for  example  the  roots  of 
iA.&  -  X  l|=  O  have  negative  real  parts)  and  G  contains  the 
or*rigin,  optimal  control  exists  for  all  Initial  states.  Since 
annythlng  which  can  be  done  v/lth  allowable  control  can  also  be 
doone  with  optimal  control,  the  synthesis  procedure  to  be  out- 
ILined  will  provide  the  domain  of  controllability,  that  is, 
thne  set  of  initial  states  from  which  the  set  G  can  be  reached 
usBing  allowable  control. 

The  necessary  and  sufficient  condition  discussed  in 
sesotlon  3  is  of  interest  in  synthesizing  the  optlmvim  feedback 
ooontroller.  The  procedure  used  is  the  familiar  one  of 
riKinning  backwards  in  time  from  points  of  the  boundary  of  the 
tajirget  G. 

I 

Starting  at  x(0)  in  boundary  of  G  with  the  extremal 
coontrol  u(t)  =  sgn{Tioe“At^j},  where  t^q  is  nomal  to  a  support 
pLlane  of  Q  at  x(0)  and  directed  into  G  we  consider  the 
ooorre spending  response  x(t)  as  t  — t,  -oo<  t  <  0.  All  such 
stsates  x(t)  which  can  be  reached  in  the  above  manner  determine 
thne  domain  of  controllability.  Moreover  along  such  optimal 
trr’ajectorles  the  control  u  can  be  specified  as  u(x),  the  re- 
qiKiired  feedback  control  function. 

To  illustrate  this  further  consider  an  example : 

•1  2 
X  =  x^  +  u 

*2  1 
rr  =  -X  +  u 

wiith  |uj£  1.  Suppose  it  is  required  that  x^(t)  go  to  zero  and 
resmaln  so  in  minimum  time.  We  see  easily  that  G  is  Just  the 
ILLne  segment;  |x®  I  Cl,  x^  =  0  ,  For  this  example  qoe-Att  = 
ilcx)^(cos  t  -  sin  t)“+  ^  +  cos  '•')  snd  the  points  of 

Innterest  are  those  on  trajectories  with  x(0)  eO  and 
uQt)  =  sgn  (Tioe"^%) which  corresponds  to  values  of  t  <  0 
whciere  ui(t)  changes  sign,  (iig  satisfying  the  transvefsallty 
ccondltlon  at  x(0)).  For  |x®(0)L  <  1,  tJq  must  be  zero  and  the 
zesros  of  cos  t  -  sin  t  are  at  ^  -  n  tt,  n  =  1,2,.,.  .  Each  of 
tbnese  values  of  t  gives  rise  to  a  mapping  of  the  line  segment 
0  (through  the  differential  equations)  onto  another  line 
sesgaent;.  These  line  segments  are  Joined  by  sets  of  arcs  of 
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circles  which  correspond  to  values  of  t  for  which  Tioe"^‘*'h 
is  zero  for  the  sets  of  t)o  allowed  at  the  points  of  0  where 
|x®(p)  1=1,  All  of  these  point  sets  when  taken  with  the  set 
Q  divide  the  phase  plane  into  two  parts.  In  one  part  u(x)  =  1 
and  in  the  other  u(x)  =  -1  is  the'  optimal  feedback  control. 
This  is  Illustrated  In  figure  1, 

REMARKS 

The  results  presented  here,  after  slight  technical 
modification  [4],  apply  equally  well  to  systems  described  by 
recurrence  equations  (sampled  data  systems). 
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